Background: The number and quality of well-designed
scientific studies in the orthopaedic literature are limited. The
purpose of this review was to determine the methodological qualities of
published meta-analyses on orthopaedic-surgery-related topics.
Methods: A systematic review of meta-analyses was
conducted. A search of the Medline database provided lists of meta-analyses
in orthopaedics published from 1969 to 1999. Extensive manual searches
of major orthopaedic journals, bibliographies of major orthopaedic
texts, and personal files identified additional studies. Of 601
studies identified, forty met the criteria for eligibility. Two
investigators each assessed the quality of the studies under blinded
conditions, and they abstracted relevant data.
Results: More than 50% of the meta-analyses included
in this review were published after 1994. We found that 88% had methodological
flaws that could limit their validity. The main deficiency was a
lack of information on the methods used to retrieve and assess the
validity of the primary studies. Regression analysis revealed that
meta-analyses authored in affiliation with an epidemiology department
and those published in nonsurgical journals were associated with
higher scores for quality. Meta-analyses with lower scores for quality
tended to report positive findings. The meta-analyses that focused
upon fracture treatment and degenerative disease (hip, knee, or spine)
had significantly lower mean quality scores than did meta-analyses
that examined thrombosis prevention and diagnostic tests (p < 0.05).
Conclusions: The majority of meta-analyses on orthopaedic-surgery-related
topics have methodological limitations. Limitation of bias and improvement
in the validity of the meta-analyses can be achieved by adherence
to strict scientific methodology. However, the ultimate quality
of a meta-analysis depends on the quality of the primary studies
on which it is based. A meta-analysis is most persuasive when data
from high-quality randomized trials are pooled.