In clinical medicine, accurate and consistent reporting of complications is critical, since their nature and frequency represent pertinent clinical information for the safety evaluations required for the development or improvement of medical procedures and devices. Documented complication data are also crucial for comparisons of implants used for the same indication1 and to help identify patient groups at risk2. Recently, the reduction of complications has been defined as an appropriate study goal for the testing of new pharmaceuticals that are claimed to accelerate fracture-healing3,4.
It is surprising that there are neither uniform definitions nor established classifications for the reporting of complications. According to the International Organization for Standardization (ISO), the definition of adverse events and their likely causes meets the requirements of regulatory bodies but not always the need of clinicians5. This leaves open the possibility for interpretation of the meaning of the term "complication" and might explain a portion of the large variation in complication rates reported by different authors working with similar implants or techniques. For example, the rate of major complications associated with the use of autologous bone grafts from the iliac crest ranges between 0.8% and 25% and the rate of minor complications ranges between 9.4% and 25%, depending on the authors and definitions6. In addition, an association between the findings and funding of scientific studies, recently reported by Okike et al.7, highlights the potential conflicts of interest involved in this issue.
We applied a group-developed complication-reporting checklist to a series of selected orthopaedic randomized controlled trials published during a two-year period. The specific goal of this systematic review was to determine whether complications were considered to be part of the study outcome and, if so, whether essential data regarding the assessment of the prevalence, severity, and characteristics of complications after various orthopaedic interventions were consistently provided by the authors. We hypothesized that reporting of complications in the surgical literature has been highly variable.
Identification of Randomized Trials
Five peer-reviewed orthopaedic journals (Acta Orthopaedica, Clinical Orthopaedics and Related Research, The Journal of Bone and Joint Surgery [American and British volumes], and The Journal of Trauma) were screened for randomized controlled trials published between January 2006 and July 2007. All of the original papers on the randomized controlled trials were obtained, anonymized, and distributed to two external reviewers for data extraction. Any disagreements between the two reviewers were resolved by consensus.
Reporting of Complications
We developed a checklist for the assessment of complication reporting in the orthopaedic randomized controlled trials. The checklist consisted of three main parts: definition, evaluation, and reporting. The principal question was whether the occurrence of complications was considered to be part of the outcome. If this was answered positively, the "definition" section was completed. The definition section included questions about anticipated complications and their definitions as well as questions about death, malunion, impaired function, and a reoperation as complications. The next question was whether the period of observation was clearly defined and was similar for all patients. The evaluation part of the checklist contained one question asking who assessed the complications and another about the rating of severity and causality. The last section of the checklist addressed information about complications necessary for the calculation of relative risk (e.g., the number of patients, fractures treated, and complication risks). The checklist was used without prior validation.
Data Analysis
The data were entered into a Microsoft Access database (Microsoft, Redmond, Washington). The analyses were descriptive with summarized results presented as proportions of the total number of trials included and continuous variables summarized as the median value.
The results for the main part of each trial were stratified as either surgical or nonsurgical depending on the reported treatments. Trials that involved a comparison between surgical and nonsurgical treatments were categorized as "surgical" because of the potentially higher impact of surgery. Surgical interventions were further categorized as fracture treatment, arthroplasty, and other interventions, and nonsurgical interventions were identified as drug treatment for pain, drug treatment for another symptom or condition, or other nonsurgical methods. The rationale for this categorization was that the primary objective of each trial could also determine the view on possible or anticipated complications.
Literature Search
One hundred and twelve randomized controlled trials were identified and considered in this review (see Appendix). Thirty-four trials were published in the American volume of The Journal of Bone and Joint Surgery; thirty-three, in the British volume of The Journal of Bone and Joint Surgery; twenty, in Clinical Orthopaedics and Related Research; twenty, in Acta Orthopaedica; and five, in The Journal of Trauma. The majority (seventy-three) of the trials involved surgical treatment methods, with most of these trials focusing on arthroplasty (38% of all trials) (Table I).
Complication Reporting
Complications were included as outcomes in two-thirds of the randomized controlled trials overall and in 93% of the trials involving surgical fracture treatment (Table II). A list of anticipated complications (i.e., those considered more specifically in the trials) was provided in only 29% of the trials in which complications were examined as a study outcome. Definitions of anticipated complications were provided at least partly in only two trials on fracture treatment8,9 and in six other trials10-15.
Although complications were considered as part of the outcome in two-thirds of the randomized controlled trials, in the majority of the trials (57% to 67% of all randomized controlled trials and 21% to 50% of the fracture-treatment trials) it was unclear whether the authors considered the specific events death, malunion, impaired function, and a reoperation as complications and defined them prior to the study as anticipated complications. Impaired function was not explicitly considered and reported as a complication in 93% of all trials, and a reoperation was considered as a complication in only 50% of the fracture-treatment trials (Fig. 1).
Assessment of Timing of Complications
The period of observation was clearly defined and similar for all patients (unless they were lost to follow-up) in 92% (sixty-nine) of all randomized controlled trials in which complications were reported and in 93% (thirteen) of the fourteen trials on fracture treatment. The period of observation was limited to up to six months in 54% (thirty-seven) of the sixty-nine randomized controlled trials and in 38% (five) of the thirteen trials related to surgical (fracture) treatment (Table III). Predefined follow-up time points for the assessment of complications were identified in three trials related to surgical (fracture) treatment, but this information was not provided in the other ten surgical (fracture-treatment) trials.
Assessors of Complications
The person or group who assessed the complications was not reported in 83% (sixty-two) of the seventy-five randomized controlled trials, and this process was carried out by the treating surgeon or physician in an additional 8% (six). The principal investigator reviewed all complications in only one trial related to pain management16. We did not identify any trial in which a data safety review board assessed and classified complications. Complications were either assessed by study nurses or self-assessed by the patients (included in the category of "other" on the checklist) in three randomized controlled trials each. Complication severity was reported in only 9% (seven) of the seventy-five randomized controlled trials and 21% (three) of the fourteen involving fracture treatment. Further assessment of potential causes and outcomes of the reported complications was inconsistent. The number of complications that occurred was reported in all but one of the randomized controlled trials, but only three trials provided information on the number of patients who had experienced at least one complication. Complication risks or rates were reported in nearly one-third (twenty-three) of the seventy-five randomized controlled trials and in seven (50%) of the fourteen trials on fracture treatment.
Despite the need for and importance of reporting complications in orthopaedic journals, no consistent standard universal reporting of these events was observed in this literature review. Anticipated complications or adverse events were well defined and categorized in only a minority of the studies that were evaluated. Because of variations in, or a lack of, classification systems for complications in the studies, it is not possible to compare different types of treatments/implants on the basis of complication rates from a methodological point of view.
Identification of complications and the reduction of their prevalence were considered to be part of the study objectives in only a few trials in this review. This is in contrast to other reports, where reduction of complications and re-interventions was a well accepted study objective17-22. Safety data should be systematically considered in all clinical studies, and if this outcome information is to be reliable and interpretable, a universal and widely accepted system for the classification of complications is required as an outcome tool. Without such a system, there is much room for individual judgment and interpretation. Orthopaedic surgeons are mainly interested in "implant-related" complications, and they seem to neglect mild problems that do not have obvious consequences. Systemic or general complications such as sepsis are rarely considered to be implant-related.
The use of an appropriate study design as well as unbiased evaluation of safety data will help to resolve certain controversial surgical issues. For instance, the ongoing discussion about the surgical risk of vertebroplasty as compared with that of kyphoplasty in the treatment of vertebral osteoporotic compression fractures demonstrates the necessity of rigorous recording of all complications independent of surgeon opinion regarding causality23.
Traditionally, many orthopaedic studies have focused on the comparison of two or more treatment modalities for one indication. If identification of implant-related complications is part of the study objective, the anticipated complications have to be clearly defined before the study is begun. Specific complications such as cutout from the proximal part of the femur24 or humerus25 and loss of reduction at the distal part of the radius26 are known from the clinical literature and can be defined prior to prospective documentation. Also, it should be verified, in a primary assessment during the study, that the surgical treatment was carried out correctly. Cases of incorrect implant placement and other iatrogenic failures cannot be considered implant failure; instead, they might be entered into a separate analysis concerning surgical skills and user friendliness.
The potential for a conflict of interest when the treating surgeon evaluates the occurrence of complications is obvious. Although some authors are convinced that negative results are published less often27,28, others have not reported that finding29. Surgeons and their industrial partners may not be motivated to report high complication rates27,28,30. Independent reviews of complications by observers blinded to the treatment and to the identity of the treating surgeon, such as a data and safety-monitoring committee, would promote accurate and complete reporting, although such reviews require additional effort and resources. A data and safety-monitoring committee in a clinical trial is a group of individuals with pertinent expertise who review, on a regular basis, accumulating data from one or more ongoing clinical trials. The data and safety-monitoring committee advises the sponsor regarding the continued safety of the intervention for the trial subjects and the safety for those to be recruited into the trial as well as the continuing validity and scientific merit of the trial31. In our review, none of the analyzed studies included data assessment by an independent committee that corresponded to a Data Safety Review Board. This finding illustrates a lack of awareness about potential conflicts of interest, with an associated detection bias, and corresponds to previous observations that industry-funded orthopaedic trials show better results than do those not funded by industry7,30,32. We believe that multicenter trials with prospective monitoring by an established data safety review board offer the best research setting for validating safety data and minimizing reporting bias.
It is also important to report exactly when a complication occurred as this information allows distinction between early and late complications as well as better comparisons between studies with similar observation periods.
Interestingly, our review revealed some uncertainty regarding what should be considered a complication in any study. While reoperations and malalignment were regarded as specific complications in most fracture-treatment studies, a severe functional deficit was considered to be a complication in only one such trial33. We believe that these events should not be considered as complications per se but as unintended consequences of complications. The indication for a reoperation after failed treatment is a therapeutic decision that is not based exclusively on the complication type and severity. Other factors such as the overall functional outcome, age, comorbidities, and the surgeon's attitude about surgical risk also influence the decision-making. For instance, a loss of function is widely accepted in older patients with distal radial fractures, while younger patients would receive a re-intervention to improve functional outcome34. Bone malalignment (varus/valgus, recurvatum/antecurvatum, rotation, or shortening) should be considered as an unintended outcome following loss of anatomical reduction. Furthermore, the relevance of a severe functional deficit may be assessed on the basis of whether it required re-intervention, which is more likely to be performed in younger patients than in older ones, but it also can be evaluated according to predefined thresholds for poor (and unacceptable) functional outcomes.
A literature review is only as good as the underlying data. Interestingly, the majority of the randomized controlled trials that we reviewed were published in one of the two major clinical orthopaedic journals, emphasizing their quality. Randomized controlled trials in orthopaedics, as compared with trials in medical specialties such as rheumatology, must undergo substantial improvements, including appropriate complication assessment and reporting.
Our approach is limited by our use of a nonstandardized, nonvalidated checklist; a validated checklist is unavailable given the novelty of the field of complication reporting. As the field evolves, the assessment of such checklist tools should be considered as a way to minimize variability in intraobserver and interobserver reliability as well as to develop a widely accepted method.
A standard protocol for assessing and reporting complications should be developed and endorsed by professional organizations, ethics committees, journal editors, and most importantly clinical investigators.
On the basis of this review, we suggest that the following be considered by those conducting future orthopaedic trials.A randomized controlled trial should be performed if possible.Complications should be included as an outcome in any clinical investigation plan.Anticipated complications should be defined prior to the start of the study.There should be a blinded assessment of the outcomes.Complications should be monitored by an independent and competent data review board.There should be a safety analysis of complications within a standard minimum observation period.
A randomized controlled trial should be performed if possible.
Complications should be included as an outcome in any clinical investigation plan.
Anticipated complications should be defined prior to the start of the study.
There should be a blinded assessment of the outcomes.
Complications should be monitored by an independent and competent data review board.
There should be a safety analysis of complications within a standard minimum observation period.
Note: The authors thank Ryan Degen and Jerzy Hubert for their help with the data extraction process for this review.