Background: The American edition of The Journal of Bone and Joint Surgery (JBJS-A) has included a level-of-evidence rating for each of its clinical scientific papers published since January 2003. The purpose of this study was to assess the type and level of evidence found in nine different orthopaedic journals by applying this level-of-evidence rating system.
Methods: We reviewed all clinical articles published from January through June 2003 in nine orthopaedic journals. Studies of animals, studies of cadavera, basic-science articles, review articles, case reports, and expert opinions were excluded. The remaining 382 clinical articles were randomly assigned to three experienced reviewers and two inexperienced reviewers, who rated them with the JBJS-A grading system. Each reviewer determined whether the studies were therapeutic, prognostic, diagnostic, or economic, and each rated the level of evidence as I, II, III, or IV. Reviewers were blinded to the grades assigned by the other reviewers.
Results: According to the reviewers' ratings, 70.7% of the articles were therapeutic, 19.9% were prognostic, 8.9% were diagnostic, and 0.5% were economic. The reviewers graded 11.3% as Level I, 20.7% as Level II, 9.9% as Level III, and 58.1% as Level IV. The kappa values for the interobserver agreement between the experienced reviewers and the inexperienced reviewers were 0.62 for the level of evidence and 0.76 for the study type. The kappa values for the interobserver agreement between the experienced reviewers were 0.75 for the level of evidence and 0.85 for the study type. The kappa values for the agreement between the reviewers' grades and the JBJS-A grades were 0.84 for the level of evidence and 1.00 for the study type. All kappa values were significantly different from zero (p < 0.0001 for all). The percentage of articles that were rated Level I or II increased in accordance with the 2003 journal impact factors for the individual journals (p = 0.0061).
Conclusions: Orthopaedic journals with a higher impact factor are more likely to publish Level-I or II articles. The type and level of information in orthopaedic journals can be reliably classified, and clinical investigators should pursue studies with a higher level of evidence whenever feasible.
The authors did not receive grants or outside funding in support of their research or preparation of this manuscript. They did not receive payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity. No commercial entity paid or directed, or agreed to pay or direct, any benefits to any research fund, foundation, educational institution, or other charitable or nonprofit organization with which the authors are affiliated or associated.
Investigation performed at the Departments of Orthopaedic Surgery at Vanderbilt University, Nashville, Tennessee; Boston Medical Center, Boston University, Boston, Massachusetts; and McMaster University, Hamilton, Ontario, Canada
- Copyright © 2005 by The Journal of Bone and Joint Surgery, Incorporated
Enter your JBJS login information below.
Please note that your username is the email address you provided when you registered.