The article “A Cost-Utility Analysis Comparing the Cost-Effectiveness of Simultaneous and Staged Bilateral Total Knee Arthroplasty” compares two types of surgical treatment of patients with bilateral knee problems: simultaneous and staged bilateral total knee arthroplasty. It is clear that patients undergoing two concurrently implemented total knee arthroplasties must spend more time in the operating room and need more anesthesia, more transfused blood, and more complex rehabilitation than patients undergoing any single total knee arthroplasty. Nevertheless, when comparing two concurrently implemented total knee arthroplasties with two consecutively implemented total knee arthroplasties, the result is not so obvious. Therefore, the topic of this article is very important from a practical standpoint: if surgeons do not observe contraindications for simultaneous bilateral total knee arthroplasty in patients with bilateral knee problems, should they recommend a simultaneous or a staged one? Most publications either do not find differences between the two strategies or show that staged bilateral total knee arthroplasties are less risky than simultaneous ones. Nevertheless, these publications rarely use a monetary expression of the risks as a common platform for their integrated comparison.
After analyzing a large secondary data set, the authors conclude that “Using the matched sample, all complication rates were higher for the staged group.…On the basis of this analysis, simultaneous bilateral total knee arthroplasty is more cost-effective than staged bilateral total knee arthroplasty” (see the Abstract). This conclusion is strong, but does it withstand the limitations of analyzed data, applied statistical models, and reported results?
To answer this question, I will briefly compare three scenarios that could be used in comparative studies of concurrent and staged total knee arthroplasties (from the most rigorous to the actual) and will evaluate their impact on the potential conclusiveness of reported findings.
Scenario 1. Multicenter Randomized Controlled Trial (RCT)
In the RCT design, consented patients with severe osteoarthritis in both knees and who are eligible for simultaneous bilateral total knee arthroplasty (as a more demanding procedure) should be randomized to receive either simultaneous or staged bilateral total knee arthroplasty. Patients who are not eligible for simultaneous bilateral total knee arthroplasty because of age or comorbid medical conditions should be excluded from the consenting and randomization. Taking into account that the prevalence of simultaneous bilateral total knee arthroplasty varies from one provider to another one, randomization should be implemented within providers. This design ensures that two study arms (simultaneous and staged bilateral total knee arthroplasty) are equivalent at baseline. Practical implementation of this scenario is questionable because of multiple factors.
Scenario 2. Observational Study Based on Secondary Data: General Design
Data for conventional observational studies should be collected as a part of routine clinical activities and should be analyzed retrospectively. Patients who underwent simultaneous bilateral total knee arthroplasty would constitute a study cohort; a matched control cohort of staged total knee arthroplasties has to be built artificially using propensity matching or any other procedure that minimizes the differences between the two cohorts at baseline. Both cohorts would contain patients with severe bilateral knee problems. The sample size would not impact the cost of the study as in RCTs, thus not limiting the sample size. The need of artificial matching brings an undesirable uncertainty because it depends on the available information and statistical preferences of the analysts. Any change in the list of available baseline variables or matching algorithms impacts the composition of the groups and the results of their comparison. There is no consensus on how to report the results of observational studies to avoid their misrepresentation. Nevertheless, it is clear that all critical assumptions have to be documented.
Scenario 3. Observational Study Based on Secondary Data: Current Study
This particular observational study used secondary data that do not support Scenario 2 because of a lack of some critical information. As a result, staged bilateral total knee arthroplasties had to be modeled as two unilateral total knee arthroplasties. The authors were creative when selecting analytic and corrective statistical procedures in this situation. It allowed them to solve many (although not all) problems related to these data. Nevertheless, when discussing limitations of this observational study, they underemphasized some critical issues.
First, most modeled staged bilateral total knee arthroplasties were represented by a chain of unilateral surgeries implemented in patients with unilateral knee problems. It is not clear to what extent it approximates actual staged bilateral total knee arthroplasties in patients with bilateral problems.
Second, the use of the term “staged bilateral total knee arthroplasties” instead of “modeled staged bilateral total knee arthroplasties” in the Results section (including tables) seems to be inappropriate; it creates a false impression that the actual staged bilateral total knee arthroplasties were analyzed.
Third, when modeling staged bilateral total knee arthroplasties, the authors did not take into account changes in patient age and medical status between the two consecutive surgeries; they did not justify this omission.
Fourth, propensity-based matching was only partially successful because the samples were significantly different on many parameters after matching. Moreover, this matching cannot be fully successful because one cohort contains patients with bilateral knee problems while the second cohort is dominated by patients with unilateral knee problems.
Summarizing, one could say that despite all the authors’ attempts to compensate for problems of the secondary data that they used, the conclusiveness of their findings is low. This study rather demonstrates that no one statistical technique (even a very sophisticated one) can completely cure the disadvantages of improperly selected secondary data. If the same analytic methodology and amount of effort were applied to a data set that explicitly identified concurrent and staged bilateral total knee arthroplasties, they would provide much more useful and convincing conclusions.