In the Eye of the Beholder
Commentary on an article by Joshua D. Auerbach, MD, et al.: “Mitigating Adverse Event Reporting Bias in Spine Surgery”
Jeffery L. Stambough, MD, MBA

Commentary

Auerbach et al. should be congratulated on the publication of their paper on mitigating bias in the reporting of adverse events in spine surgery. Equal praise is deserved by Paradigm Spine, the corporate sponsor of the Level-I, prospective, randomized multicenter study of the coflex interlaminar stabilization device, for incorporating a Clinical Events Committee (CEC) for the readjudication of the study’s adverse events and device-related complications end point.

It has been said that “the data is the data.” The rub is often in the interpretation and presentation of those data, which are complicated by a wide variety of biases—both perceived and real. This process is even more likely exposed to biases when subjective rather than objective ratings are involved. The primary issue addressed by Auerbach et al. is the fact that adverse events and complications tend to be underreported by the principal investigator or those intimately involved in the study, who may or may not have additional financial or other conflicts of interests. Certainly, there seems to be a human tendency to minimize the bad and maximize the good. This does not imply malice or dishonesty. Regardless, this tendency is a source of observational bias1.

In this study, the CEC members, three independent blinded spine surgeons, were commissioned to recategorize adverse events into one of three severity categories (mild, moderate, severe) and also to reassess the relationship of reported complications to the device or surgery in one of five categories (unrelated, unlikely, possibly, probably, or definitely). These are subjective rankings, although numerical values could be applied. Furthermore, these independent spine surgeons were knowledgeable, were trained, and had no financial interest in or relationship to the study sponsor or investigators. These physicians reviewed the complications in one day and repeated the process about two years later, demonstrating that their subjective ratings were consistent and reliable over that period of time.

Thirty-seven percent of the reported adverse events and complications were reclassified by the CEC. The vast majority of these, 89%, were upgraded. Of those upgraded in relatedness, about 10% to 20% moved from the lower three rankings (unrelated, likely, possibly) to a probable or definite ranking. It is important not to extrapolate or infer any significance of this fact. It simply means that these independent evaluators were stricter on average in their rankings. It does not mean that the data were different, nor does it imply clinical significance, much less a good or bad interpretation. Interestingly, potential financial conflicts of interest were not demonstrated to be a factor in the reassignment or readjudication of adverse events and related complications in this study.

The concept of an independent, blinded evaluator is not new, nor is the use of a CEC concept. The authors rightly recognize that this is apparently the first time that reporting bias has been analyzed in the setting of an industry-sponsored device trial with this degree of rigorous statistical evaluation2,3. The use of independent blinded evaluation, specifically by a CEC, is worth further application, corroboration, and correlation, specifically when it comes to industry-sponsored or third-party-sponsored research.

The bottom line is that a CEC concept seems like a suitable and needed checkrein for third-party-sponsored studies. A CEC provides an independent, impartial perspective that may differ substantially from those of individuals more directly involved in the industry-sponsored study4. The authors are careful not to imply any clinical significance of the readjudication observed in this study. However, it is noteworthy that the three members of the independent blinded committee had a different perspective regarding the severity and relatedness of adverse events in about 37% of such occurrences. The goal of reducing the effect of investigator bias, especially in such ratings of an observational and subjective type, is valid. These findings likely apply to objective findings as well and help encourage progress toward the ultimate goal of openness and transparency in research. The use of a CEC as well as other methods that aim to decrease conflict of interest, bias, and other inherent limitations necessitate further investigation. Unfortunately at this time, industry-sponsored studies still remain inherently flawed by their very nature5.

Footnotes

  • *The author did not receive payments or services, either directly or indirectly (i.e., via his institution), from a third party in support of any aspect of this work. He, or his institution, has had a financial relationship, in the thirty-six months prior to submission of this work, with an entity in the biomedical arena that could be perceived to influence or have the potential to influence what is written in this work. The author has not had any other relationships, or engaged in any other activities, that could be perceived to influence or have the potential to influence what is written in this work. The complete Disclosures of Potential Conflicts of Interest submitted by authors are always provided with the online version of the article.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.