Background: Investigators aim to publish their research papers in
top journals to disseminate their findings to the widest possible audience.
Systematic reviews of the literature occupy the highest position in currently
proposed hierarchies of evidence. We hypothesized that the number of citations
(a measure of scholarly interest) for systematic reviews (or meta-analyses)
published in leading orthopaedic journals would be greater than the number of
citations for narrative reviews published in the same journals.
Methods: We identified fifteen journals that had high Science
Citation Index impact factors for the orthopaedic subspecialty and were
believed to have a higher yield of studies and reviews of scientific merit and
clinical relevance. For the year 2000, six research associates applied
methodological criteria to each article in each issue of the fifteen journals
to determine whether the article was scientifically sound (rigorous versus
nonrigorous). Of the 3916 articles identified, 2331 were original or review
articles. We queried the ISI (Institute for Scientific Information) Web of
Science database to ascertain, as of March 2003, the number of subsequent
citations to each one of the reviews after its original publication in all
journals that published both narrative and systematic reviews.
Results: Of the 2331 articles published across the fifteen journals
in the year 2000, 110 were review articles. Only seventeen (15%) of the 110
reviews met our criteria for systematic reviews with rigor. Rigorous
systematic reviews received more than twice the mean number of citations
compared with other systematic or narrative reviews (13.8 compared with 6.0, p
= 0.008). The rigor of a review was a significant predictor of the number of
citations in other orthopaedic journals (p = 0.01). In addition, rigor was
significantly associated with the number of citations in nonorthopaedic
journals (p = 0.03).
Conclusions: Our findings suggest that journal editors and authors
can improve the relevance and scholarly interest in their reviews (as shown by
the number of citations) by meeting standard guidelines for methodological