To The Editor:
We read with interest and concern the article, "Navigated Total Knee
Replacement. A Meta-Analysis" (2007;89:261-9) by Bauwens et al. We
submitted a similar meta-analysis to The Journal of Bone and Joint
Surgery over one year ago, which was appropriately rejected for
publication because of the inclusion of data from abstracts and uncontrolled
case series. The reviewers and editors also expressed concern that our finding
of an advantage for navigated total knee arthroplasty compared with
conventional total knee arthroplasty based on radiographic alignment end
points needed to be balanced against the lack of evidence with regard to
differences in cost-effectiveness, complication rates, and long-term outcomes
between the two procedures.
We were in the process of updating our meta-analysis in light of more
recent publications (excluding data from abstracts and uncontrolled case
series) when the study by Bauwens et al. was published. Having reviewed
essentially the same database, we were perplexed by the authors' conclusion
that "navigated knee replacement provides few advantages over
conventional surgery on the basis of radiographic end points," as our
own meta-analysis revealed a significant improvement in radiographic end
points with computer-assisted navigation.
Our concerns about the discrepancies between our findings and those of
Bauwens et al. prompted us to investigate their source data. We contacted
them, and they graciously provided us with the raw data for all studies
included in their meta-analysis. On further review, we discovered multiple
inaccuracies of data extraction and/or data entry in their analysis.
In four of the studies1-4 reviewed in the article by Bauwens et
al., the data for conventional techniques were entered into the data set for
navigated replacement for analysis while the data for the navigated
replacements were entered into the data set for conventional techniques. We
were also able to determine errors of data extraction, data entry, patient
count, or patient group assignment from four additional studies5-8.
One paper9 was included and counted as reporting mechanical axis
data when these data were not reported in the study. A kinship
study10 (i.e., a study sharing overlapping data with an already
included study) was included when it should have been excluded. There were two
additional studies11,12 in which the numbers that we extracted were
slightly different from those in the report by Bauwens et al.; we note these
only as discrepancies (not errors) in extraction.
Our further review of their paper also suggested that their labeling and
description of results were misleading. Specifically, they describe their
meta-analyses as those of "relative risk of malalignment" and
label their figures accordingly. In the Discussion, they state that "the
available data suggest that navigation reduces the relative risk of 3° of
malalignment by 25%." This statement is in error because their
meta-analysis was not of the relative risk of malalignment, but rather the
relative risk of alignment (i.e., the chance that a patient has alignment
after the procedure). It would, therefore, have been accurate for them to
state that conventional total knee arthroplasty decreases the relative chance
of alignment by 25%. When misfit, instead of fit, is the outcome of choice,
the results are quite different from those reported by Bauwens et al.
Correctly stated, the risk of malalignment with conventional replacement is
appropriately three times that with computer-assisted surgery.
In conclusion, our findings of data extraction and entry errors cause us to
challenge the conclusions in the article regarding the meta-analysis of
radiographic end points following conventional compared with navigated knee
replacement surgery. A correct data analysis demonstrates overwhelming
evidence of a much lower error rate with navigation. Reversal of some of the
extracted data and misreporting of relative risks for fit as risks of
malalignment are partially responsible for the muted difference that Bauwens
et al. described between navigated and conventional total knee arthroplasty.
These errors, however, do not obviate their other discussion points regarding
the methodological limits of the available trials, including a dearth of
evidence on long-term outcomes, quality of life, and costs.
While we recognize and understand the challenges inherent in performing
meta-analyses, our intent is to bring these errors to the attention of the
readers of The Journal to correct any erroneous impression that this
work may have left with the readership.