To begin developing the ICOR, a conference was held on May 9 and 10, 2011, at the headquarters of the FDA in Silver Spring, Maryland. The ICOR conference summarized the existing international data sources and methods for post-market evaluation and surveillance of orthopaedic devices. The conference invitees included seventy-three stakeholders from twenty-nine total joint arthroplasty registries representing fourteen nations. In addition, more than twenty-five non-registry stakeholders represented the medical device industry, the Agency for Healthcare Research and Quality (AHRQ), the National Institutes of Health (NIH), CMS, academia, device regulatory agencies, device cataloging experts, insurers, and other payers. The meeting was the initial primary step in building the ICOR methodological infrastructure for evaluating orthopaedic implant safety and effectiveness.
The broad participation of national and international registries at various stages of development ensured a comprehensive approach toward development of the ICOR. All stakeholders expressed the need for infrastructure that would permit them to collaborate and focus on understanding the variability in outcomes of total hip and total knee arthroplasty. The critical and recurring theme was the need for coordinated registry efforts centered around collecting, measuring, and harmonizing data, including the challenge of describing and classifying the devices captured in the ICOR registries.
The combined ICOR registries will include data on more than 3,500,000 orthopaedic surgical procedures, and will capture all implantable devices on the market. The consortium of registries is committed to filling the gaps in our knowledge in order to support clinicians, orthopaedic device regulators, payers, patients, and industry. Using innovative and harmonized methods along with data from registries, ICOR registries can collaborate to answer priority research questions about device safety and effectiveness.
Setting the Stage
The meeting opened with an introduction by the organizers, who emphasized the need for the ICOR to advance collaborative registry-based research by building the infrastructure and methodology for evidence-based regulatory science. The FDA provided an overview of the CDRH's mission, which includes responsibility for device classification, the 510(k) decision process, the premarket application (PMA) process, investigational device exemption (IDE) regulations, and preapproval studies; this was followed by a discussion of possible uses of registry data. A panel discussion focused on implementation of unique device identifiers (UDIs) and on how the ICOR can help the FDA to catalog the attributes of implants. The values of linking registry data to other sources, building consortiums and infrastructure for funded studies, and methodological innovation were emphasized.
Summary of the Session on the Current Status of International Collaborations
International collaborations involve a series of initial steps to clarify data definitions, data collection procedures, and methods for making meaningful comparisons between interventions. Most collaborations involve thorough assessment of the clinical heterogeneity of patient populations, the specifics of the interventions, and outcome measurement in a particular country.
Two major collaborative efforts among ICOR registries are already in existence: (1) the Nordic Arthroplasty Register Association (NARA), and (2) a collaboration between the Kaiser Permanente National Total Joint Replacement Registry and the Norwegian Arthroplasty Register involving knee arthroplasty.
The NARA collaboration involves direct sharing of data by three Scandinavian countries (Sweden, Denmark, and Norway) that have agreed to harmonize their data definitions and data collection. A fourth Scandinavian country (Finland) has recently joined this group. These countries have a long history of registry data collection, including a wide variety of relevant exposure data and health outcomes data in clinically heterogeneous populations. The NARA research leadership highlights the value of feedback to surgeons, which helps them to implement best practices. Key factors contributing to NARA's success include participation of relatively small countries with similar health care systems, the existence of nationwide registries with a high degree of coverage, and universal personal identifiers that allow patient tracking.
The practice of surgery varies among the NARA countries. For example, the implant fixation method (cemented compared with cementless) and the surgical approach used in hip arthroplasty vary from country to country. A combined NARA case study (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.00951) indicated inferior outcomes following hip resurfacing compared with total hip arthroplasty. In addition, the type of fixation and an age of <50 years were associated with a greater risk of revision knee arthroplasty7. NARA will work with the ICOR to harmonize terminology, standardize methods, and assure data quality in other participating registries.
The collaboration between the Kaiser Permanente registry and the Norwegian registry demonstrates the potential for collaboration between U.S. and European registries after extensive harmonization of methodology (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.01045, http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.00905)8,9. The investigators in this collaboration measured and accounted for heterogeneity of patients, treatments, and outcomes in real-world clinical settings in both countries. They created a comprehensive set of harmonized patient, treatment, and outcome variables and then compared outcomes after knee arthroplasty. Differences in outcome associated with patient characteristics, implants, and surgical techniques were detected. Opportunities for future research were identified, including comparison of mobile and fixed-bearing knee implants, comparison of fixation methods, investigation of the effect of surgeon and hospital volume, and investigation of the effect of patellar resurfacing.
Registries within the ICOR also have experience in collaborating with regulatory agencies. For example, the National Joint Replacement Registry in Australia has been collaborating with Australian regulators and the FDA, and the National Joint Registry in England and Wales has been collaborating with the United Kingdom's regulatory agency (the Medicines and Healthcare products Regulatory Agency [MHRA]). A recent evidence-synthesis initiative by the FDA includes working with the Australian registry and the Kaiser Permanente registry to explore the potential for synthesizing the data from various sources.
These successful collaborations serve as important pilot programs and showcase the potential of the ICOR to facilitate and further enhance existing collaborations. Many national registries are working with their countries’ national or multinational (European Union) regulatory agencies, so it is reasonable to consider integrating the work of the registries into the regulatory process and thus advancing the regulatory science. The ICOR can serve as a venue to expand the collaborations worldwide on a study-by-study basis. The conditions for collaboration clearly include standardization of definitions; development of common data sets when necessary; standardization of implant classification; and development of rules regarding authorship, data ownership, data sharing, and assignment of responsibilities for data analysis.
Summary of the Session on the Goals of U.S. Surgeons and Complementary Roles of the American Joint Replacement Registry (AJRR) and the ICOR
Registries have an important role within their own organizations as valued tools for in-house surveillance, quality improvement, and cost-effectiveness projects. In the absence of a national U.S. registry, hospital and regional registries serve as important tools for outcome improvement. The ICOR participants discussed hospital incentives for building registries, such as the ability to develop quality measures (e.g., to monitor compliance with standards for antibiotic timing and for prophylaxis against deep vein thrombosis) and cost containment. It was noted that use of electronic medical records (EMRs) in physicians’ offices in the U.S. would facilitate office-based data collection and reporting and thus facilitate development of a U.S. registry. All registry participants agreed that registry data should be protected from access by the medicolegal system. Many registry participants agreed that previously defined goals of 90% coverage and 90% participation after five years are unrealistic for orthopaedic surgeons in the U.S. and many other countries.
The FDA previously funded a project that laid the groundwork for a national infrastructure by identifying and assessing the existing orthopaedic implant registries in the U.S. that could participate in the creation of a national research network10. The final report included a thorough summary of the nine largest and most modern registries in the U.S. The views of the ICOR conference participants and the conclusions of the FDA report were similar: there is a need for harmonization of existing U.S. registries, as the registries are very heterogeneous in their approaches and in their goals. The American Joint Replacement Registry (AJRR) was recently launched after surgeons, companies, the U.S. government, and lawyers were consulted and the models for registry building were thoroughly discussed. The AJRR plans to have useful registry data available in 2014. However, at present there remains a huge evidence gap pertaining to device safety and effectiveness, and the ICOR network can help surgeons and other stakeholders to fill this gap. The ICOR and the AJRR are complementary initiatives and can collaborate at a variety of levels. The ICOR network can help to bring multiple disparate registries together, discuss harmonization of the data elements, and develop methodologies such as models of collaboration that do not include data sharing.
Summary of the Session on Evidence Needs and Generation from Manufacturer, Funder, and Payer Perspectives
The fact that orthopaedic devices are among the most commonly used medical devices in the U.S. has substantial cost implications. The main areas of discussion in this session were related to the benefits of and barriers to a public-private partnership between the ICOR and stakeholders. The participants agreed that the benefits, including opportunities for research to advance public health in general, substantially outweighed the risks. Some critical benefits include:
- Information that can assist in balancing quality improvements and their costs
- Development of evidence from many countries that have already adopted new devices, which will assist regulators in the device approval process in the remaining countries
- Identification of early indicators of device failure in the post-market setting
- Comparison of the cost and effectiveness of different devices, which will be useful for group purchasing and contracting
- Development of information regarding patient care, which will help to address actionable gaps in care
- Harmonization of data, which will facilitate post-market surveillance efforts
- Development of new methods to answer research questions more quickly and efficiently
An important barrier to an ICOR-stakeholder partnership is the cost of registry maintenance, which is burdensome in many countries. Developing public-private partnerships with stakeholders will assist registries worldwide. Several legal, privacy, and informed-consent requirements also need to be addressed, but these were not considered to be substantial barriers since the ICOR will not be dealing with data collection. The main concern expressed by the manufacturers involved standardization of the classification of implants, and this concern was shared by all of the registries. An additional concern of the manufacturers involved application of evidence on implant performance in one country to make regulatory decisions in another country if the results from multiple countries are conflicting. The ICOR plans to thoroughly investigate these controversies and provide a multinational answer, which will represent a welcome initiative from the industry's perspective.
Summary of the First Session on Implant Safety and Effectiveness (the Metal-on-Metal Experience)
Since their recent reintroduction into the market, use of metal-on-metal bearings has been evaluated by many surgeons because these bearings were thought to reduce implant wear and to allow use of larger femoral heads, thus reducing the risk of dislocation11,12. Metal-on-metal bearings were adopted by many surgeons and were often used even in the elderly—one out of three elderly patients treated with total hip replacement in the U.S. received a metal-on-metal implant13. However, this practice has been under scrutiny since the regulatory agency in the United Kingdom (the MHRA) alerted the public about occurrences of dangerously high metal ion concentrations in soft tissues caused by metal ion release from the implants14, and the British Orthopaedic Association subsequently developed specific recommendations15. Furthermore, DePuy recalled over 93,000 ASR metal-on-metal implants in 201016,17. The implant recall received widespread coverage in The New York Times18, and general medical journals such as the BMJ addressed various issues related to device regulation, clinician involvement, and the need to advance the evidence gathering regarding implant performance17,19.
The FDA issued 522 orders on May 6, 2011 (two days before the ICOR meeting), requesting that manufacturers of metal-on-metal total hip replacements conduct post-market surveillance of patients and answer questions pertaining to the types and rates of adverse events, metal ion levels at baseline and over time, the association of demographic and clinical characteristics with metal ion levels, and the modes and causes of implant failure. The FDA has been closely monitoring reports of hip implant failure associated with various bearing surfaces, and in November 2009 the agency initiated a series of comprehensive evaluations and syntheses of the evidence to quantify the safety and effectiveness of approved implants. The CDRH at the FDA reported that there is insufficient information on the prevalence or incidence and the natural history of local and systemic adverse reactions to metal debris in patients treated with a metal-on-metal hip replacement. The FDA is seeking information on patient characteristics that predict the development of an adverse reaction to metal debris (ARMD). It is not known whether the production of metal debris is dependent on the device design. There are also unresolved issues related to agreement among measurements of metal ion levels made by various methods and in various laboratories. The interpretation of metal ion levels is also not standardized, and there is insufficient information about the usefulness of metal ion measurements and about the relationship of metal ion levels to either local ARMD or systemic complications.
The researchers from Rush University Medical Center discussed the benefits of measuring metal levels in serum and urine. They also cautioned that routine measurement of ion levels is not recommended. Adverse local tissue responses associated with elevated serum metal concentrations might reflect accelerated wear and/or corrosion. The group also agreed with the FDA report that threshold values for local and remote metal toxicity have yet to be determined. The decision to revise a metal-on-metal implant should be based on signs and symptoms and on serum test results, and further work is needed to standardize metal testing of serum and urine.
The researchers from the National Joint Registry in England and Wales gave a brief overview of findings (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.01107)20 that led to a recent MHRA alert14. The ASR implants were introduced in the U.K. in 2005, and adverse events were first noted in a 2006 report by the registry. In April 2010, the rate of adverse events associated with the ASR implant was identified as an outlier by the registry and was reported to the MHRA. An MHRA device alert was issued in July 2010, and the ASR device was withdrawn from the market by DePuy in August 2010. There is also evidence from the registry that metal-on-metal implants are associated with a higher revision rate compared with other bearing articulations20, but the results appear to be implant-specific and longer follow-up is needed to clarify these findings.
In contrast, the researchers from the Australian Joint Replacement Registry provided very strong evidence that the use of metal-on-metal articulations in conventional total hip arthroplasty is associated with a much higher failure rate compared with other articulations (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.01220)21. This effect was consistent across all age groups and was most pronounced for implants with a head size of >32 mm and in female patients. In addition, use of metal-on-metal articulations in hip resurfacing was associated with a higher rate of revision compared with other articulations when the head size of the implant was <50 mm, in patients over sixty-five years of age, and in female patients.
These results were also supported by researchers from New Zealand's National Joint Register, who found a higher revision arthroplasty rate for metal-on-metal articulations (especially those with larger head sizes) compared with metal-on-polyethylene articulations21. Although a similar result was not found when the revision rate of metal-on-metal articulations was compared with those of ceramic-on-ceramic and ceramic-on-polyethylene articulations, the sample size of the New Zealand registry is much smaller than those of the Australian and U.K. registries. In addition, the overall revision rate in this registry is not as high as that in other registries, which may be related to a more conservative threshold for revision among New Zealand surgeons.
Finally, researchers from the Kaiser Permanente registry reported that the use of alternative bearing surfaces has increased over time within the Kaiser Permanente system. Alternative bearings such as metal-on-metal are used in younger, healthier patients. Risk factors for revision of a total hip arthroplasty in the Kaiser Permanente registry included patient and implant characteristics, such as a diagnosis other than osteoarthritis, white race, and a femoral head size of <36 mm. The metal-on-metal nature of the bearing surface was not found to be associated with a greater chance of revision in the Kaiser Permanente registry; this may be related to a shorter duration of follow-up and lower usage of ASR or similar devices within the Kaiser Permanente system.
In summary, the reports presented at the ICOR conference highlighted the controversies regarding metal-on-metal articulations, including those regarding metal ion measurements and conflicting findings from national registries. The ICOR is expected to play an important role by revealing the existence of such discrepancies between registries, addressing the heterogeneity in definitions, and developing advanced methodologies to generate knowledge that can be generalized among countries.
Summary of the Second Session on Implant Safety and Effectiveness (Fixed Compared with Mobile Bearings for Knee Replacement)
The original LCS (low contact stress) mobile-bearing implant (DePuy) was developed to reduce wear, reduce the revision rate, and improve functioning after total knee arthroplasty. This device originally had two principal designs: meniscal-bearing and rotating-platform. The rotating-platform design was the more widely used because of the greater frequency of dislocations associated with the meniscal-bearing design. Currently, many new mobile-bearing designs are available, and the safety and effectiveness of mobile bearings in general and of the individual designs have not been well studied.
Several investigators at the ICOR meeting presented findings regarding the safety and effectiveness of mobile-bearing implants (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.00982)22. The findings were consistent; most prospective randomized trials, registry reports, and meta-analyses showed no evidence of the superiority of mobile-bearing designs compared with fixed-bearing designs. For example, no benefits of the LCS mobile-bearing implant were observed within the Kaiser Permanente registry. The LCS implant was associated with a greater revision rate compared with contemporary fixed-bearing implants. Similar results were found in the Norwegian national registry. The Kaiser Permanente researchers also reported that the reasons for revision of the LCS implant were often not clear and that aseptic loosening, instability, pain, and stiffness were often correlated. The dislocation of mobile-bearing total knee implants seems to be a unique phenomenon related to this bearing design, and this subtle instability may present as pain or stiffness.
Conflicting results were presented regarding patellar resurfacing during total knee arthroplasty. Whereas the Kaiser Permanente data indicated a lower revision rate in association with resurfacing of the patella, the Norwegian registry data did not indicate such an association. Instead, the Norwegian data, which involved longer-term follow-up (more than ten years), indicated a higher rate of revision due to wear and loosening in association with patellar resurfacing. Investigators from the Swedish national registry, which has a very long duration of follow-up (more than fifteen years), confirmed the results seen in the Norwegian registry.
Summary of the Potpourri Session on Implant Safety and Effectiveness
Registry representatives reported safety concerns involving a variety of implants. A study performed by researchers from the National Joint Replacement Registry in Australia indicated slow diffusion of new implants into clinical practice (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.00867). The authors also found that none of the newest implants were associated with better outcomes, and approximately 30% of the newest implants were in fact associated with worse outcomes, compared with older standard-of-care implants23. These important findings indicate that the current approach to device development needs to be re-evaluated. Outcomes-based rather than biomechanics-based evidence should guide development of new devices.
Researchers from the European Arthroplasty Register (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.01108) indirectly substantiated the findings presented by the Australian registry researchers24. They found that many new implant safety results contained in published reports were not reproducible in the registries.
Investigators from the Netherlands re-emphasized a paradigm of phased introduction of new devices into the market, with thorough analyses at each phase (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.00907)25. The investigators also summarized the evidence supporting the use of RSA (radiostereometric analysis) and reported aggregate results from major registries regarding RSA. The authors found a 22% to 35% reduction in the number of revisions of knees tested with use of RSA during total knee arthroplasty compared with knees that were not tested. They concluded that RSA is a good qualitative tool and that its use can lead to better patient care and reduced costs.
General safety and effectiveness issues regarding implants were highlighted in three other studies. Researchers from the HealthEast registry reported good results and cost savings associated with the use of well-designed all-polyethylene tibial components in primary total knee arthroplasty. Researchers from the Register of Orthopaedic Prosthetic Implants in Italy and the Hospital for Special Surgery (HSS) presented safety studies regarding hip implants with modular ceramic necks. Squeaking was reported in 1.9% of the hips in the HSS registry.
Many researchers agreed that the implants themselves account for a substantial portion, but only a portion, of the variability in outcomes. Another important factor that should be taken into account is the surgeon's skill. The FDA's conceptual framework for evaluation of device outcomes also emphasizes the importance of surgery-related factors6. Surgery-related factors as well as device and patient factors should be part of a device's safety assessment.
Summary of the Session on Patient-Reported Outcomes (PROs): Levels of Data and PRO Measurement
Reports from a number of registries highlighted the current trends in selective inclusion of PRO measures in the registries (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.01021). For instance, the Swedish Hip Arthroplasty Register requires inclusion of PRO measures such as the visual analog scale (VAS) pain level, EuroQol-5D (EQ-5D), and VAS satisfaction, and it has a high response rate for these measures. Similarly, the New Zealand registry routinely collects PROs (Oxford scores) and found evidence that the PROs could serve as predictors of the need for revision surgery after joint replacement. Two regional registries, the Winnipeg Regional Health Authority Joint Replacement Registry and the Hospital for Special Surgery registry, routinely attempt to collect PRO data on every surgical patient, including Short Form-36 (SF-36), SF-12, EQ-5D, and Oxford-12 scores. Most registries agree that combining traditional outcome measurements with level-3 data such as PROs provides a more detailed and complete assessment of surgical outcomes. The experience of various registries with PRO use and the most commonly used PRO measures were summarized as a first step in the process of harmonizing PRO data collection among the ICOR registries26.
Summary of the Device Identification and Classification Session
Improving the safety of orthopaedic devices and assessing their effectiveness requires accurate classification of the devices. For regulated drugs, the FDA uses a National Drug Code (NDC) that specifies important drug features such as the active ingredient, manufacturer, form, strength, and packaging. In contrast, there is no universally accepted nomenclature for medical devices; instead, various stakeholders have developed disparate ways to classify devices, such as Product Codes, the Universal Medical Device Nomenclature System, and the Global Medical Device Nomenclature (GMDN) system. Orthopaedic registries have also created their own classifications that are not currently harmonized with each other. The ICOR researchers discussed the potential for harmonization of orthopaedic implant classification (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.00990)27. Some critical issues that the researchers highlighted include historical changes in prosthesis designs that were not accompanied by device name changes and changes in device names over time that did not correspond to substantial design modifications. When attempting to use catalog and lot numbers to create a classification, one should be mindful of the lack of standardization of these numbers and their lack of categorization into attributes and characteristics. The researchers supported one of the ICOR's main goals, which is the development of a universal implant classification system that would enhance worldwide collaboration in total joint arthroplasty research. The orthopaedic industry representatives at the ICOR meeting also supported this effort and pledged to supply any information needed to achieve this goal. This effort supports implementation of unique device identifications (UDIs); linking the UDI number to a clinically sound classification system will provide critical policy importance to this effort.
Summary of the Methodologies for Surveillance and Tools for Multinational Collaborations Within the ICOR
One of the main concepts of the ICOR is the use of a distributed analysis model for multinational investigations. No registry is expected to share the underlying data with any other registry. Studies can use the ICOR to address the real-world performance of devices, safety and effectiveness outcomes, off-label use and associated outcomes, the performance of subgroups, learning curve effects and the effectiveness of training, the outcomes of revision surgery, and the effects of product changes, as well as how registry studies are more suitable for studying multiple comparators (such as comparing more than ten different implants simultaneously). In addition, a model involving multiple levels of collaboration was proposed (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.01141)28. Examples of increasing levels of collaboration include clarifying information within annual reports, sharing subgroup-level effectiveness information so that results from different registries can be combined in a meta-analysis, and sharing information through the DELTA (Data Extraction and Longitudinal Time Analysis) system, which has the capability to conduct real-time surveillance of medical device outcomes29. Pilot studies have been conducted to illustrate meta-analysis of data from multiple registries and use of the DELTA system.
Researchers from the Scottish arthroplasty registry discussed the application of cumulative-sum-of-outcomes (CUSUM) methodology to identify unusual occurrences of complications, including poor performance by an implant, in real time (http://jbjs.org/article.aspx?doi=10.2106/JBJS.K.01010). CUSUM is a sequential statistical analysis technique that provides excellent visual impact when unusual occurrences are displayed graphically. It allows rapid identification of an elevated complication rate of any etiology at the surgeon level30.
The Immediate Potential for Conducting Research Projects Within the ICOR
The individual registries within the ICOR have different levels of maturity, but there are a number of research projects that can be pursued immediately. Three such projects have been identified as high priority: (1) comparative evaluation of various bearing surfaces used in hip replacement, (2) comparative assessment of various head sizes used in hip replacement, and (3) comparative assessment of fixed and mobile bearings used in knee replacement.
The ICOR participants shared information via an internal web-based survey regarding the availability of specific data within their registries (Figs. 1 through 7). More than 80% of the existing total joint arthroplasty registries collect the core data elements that are necessary for post-market surveillance of medical devices. The commonality of these data elements provides a unique opportunity for international collaboration. The ICOR coordinating center is tasked with developing the research proposal in collaboration with ICOR registries. The proposal will allow participation of most registries that have data related to implants. The contribution of large, high-quality registries will be critical for detailed and comprehensive assessment of medical device technologies. However, all registries can provide information about device utilization and specific findings relevant to technology evaluations.
Note: The authors thank the FDA leadership for support and assistance. The ICOR Planning Team consisted of Art Sedrakyan, MD, PhD (project lead, Cornell, FDA, HSS), Elizabeth Paxton, PhD (co-lead, Kaiser), Charlotte Phillips (HSS), Tadashi Funahashi (Kaiser), Timothy Wright, PhD (HSS), Thomas Barber, MD (Kaiser), Robert Namba, MD (Kaiser), Thomas Sculco, MD (HSS), and Douglas Padgett, MD (HSS). Logistical support was provided by Ilsa Klinghoffer (HSS), Allison Grande (HSS), Betty Jo Alfstad (Kaiser), and Tamia Woodruff (FDA). The authors especially thank Professor Stephen Graves and Professor Ove Furnes for leadership and support; Dr. Vernon Tolo, Editor-in-Chief of JBJS, and other editors for their support of this supplement and their careful and thoughtful management of all manuscripts; and Cathy Griffin, Editorial Supervisor at JBJS, for her assistance and oversight throughout the production of the supplement.