It is important to have processes to enforce mutually agreed-upon rules that allow data quality evaluation, checks, various manipulations, and analyses within ICOR. It is an environment where all parties are participants and are able to lead the projects with the support of a data coordinating center. The coordinating center will support each registry site, write and share computer programs, help run the computer codes, and share the results with the project clinical leaders. Orthopaedic leaders from participating ICOR registries will provide leadership, clinical expertise, and oversight for the initiative and will work with the coordinating center to provide clinical input for the analyses of the data, aggregation of the data in a way that is agreeable to all parties, and writing of manuscripts for publication in peer-reviewed medical journals.
One of the advantages of ICOR as a distributed research network is the potential to build collaborations with use of different tools at different stages of research work. The coordinating center will adopt a staged approach to help build trust and confidence among the partners. The center will help evaluate the need for technical support, resources, and meetings as collaboration is progressing. It is important to note that a lower stage does not mean an inadequate answer to safety and effectiveness concerns and a higher stage does not necessarily imply a higher level of collaboration. The tools used at various stages might be sufficient to address certain safety and effectiveness concerns. In this paper, we sought to provide an overview of the tools and stages within ICOR for designing multinational collaborative studies.
Stage I: Sharing and Clarifying Published and Unpublished General Information
This collaboration is mostly applicable to systematic reviews of evidence initiated by ICOR partners or the FDA. The process of systematic evidence review involves a comprehensive search of the evidence in MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials. Randomized clinical trials are infrequent in orthopaedics and often have many limitations due to rapidly changing technology as well as the need for large numbers of patients with long-term follow-up. In addition, clinical trials in orthopaedics are conducted in relatively unique environments defined by skilled surgeons operating at high-volume centers and often concentrating on clinical and radiographic indirect measures of device safety and effectiveness. In general, this is a reflection of the current state of research in an implantable device setting, where comparative trials are rare and often not very applicable6.
Large registries or networks of registries capturing a variety of orthopaedic devices are particularly important for comparative outcomes evaluation and active surveillance. Given long-term-evaluation requirements, only large, longitudinal, multinational registries can provide denominator data for adverse events related to specific implants and allow proper conduct of comparative safety and effectiveness studies. The ICOR researchers can collaborate with the authors of published studies and annual reports to clarify the information in the papers. The researchers can learn about outcomes that have not been reported because of space limitations related to journal publications and annual reports, or other reasons unrelated to the need to protect the information. For example, as a result of space limitations related to journal publications and annual reports, some registries concentrate on a specific technology each year and the most recent annual report does not contain the information that was reported in the previous years7.
The information obtained at this stage can be quite heterogeneous. Some registries report implant safety, assessed on the basis of the need for revision surgery, as five or ten-year survival and hazard ratios (for comparative studies), while others use the revision rate per component year of follow-up. In addition, patient demographics, diagnoses, and statistical analyses often differ greatly among the registries, precluding quantitative analyses or combining of results. In addition, the results are statistically adjusted on the basis of different covariates. However, one can still use some creativity and provide a qualitative summary of the results that can assist decision-makers (Fig. 1).
Stage II: Sharing Relevant Subgroup-Level Performance Information and Documenting Differences at the Aggregate Level Among the Data Holders
Another method for collaboration is through a combined aggregate and subgroup-level approach using tables with summary-level data and implant survival function by subgroup strata. In order to apply this approach, definitions must be first harmonized across registries. It is also important to clearly document the differences among the participating registries with regard to collection of operative and device data such as personnel (i.e., surgeons, fellows, nurses, or research personnel) and timing of operative information collection (preoperative, intraoperative, or postoperative) and follow-up. In addition, preoperative and postoperative outcome-collection modes (e-mail, mailed questionnaire, or automatic capture via electronic forms in the integrated delivery system) need to be documented. Currently, there are many differences in documentation among ICOR registries. For example, the American Society of Anesthesiologists (ASA) classification is assigned by the surgeons in the Norwegian registry, whereas this classification is determined by anesthesiologists in the Kaiser Permanente registry8. Thus, important steps for this collaboration include:
- establishment of requirements for implant safety surveillance analyses
- harmonization of definitions across registries and agreement on a common data model
- documentation of differences in outcome collection and follow-up achievements
- distribution, by the coordinating center, of an analysis plan to each participating registry (this is repeated as required on the basis of protocol modifications or evolution of the analytic plan)
- execution of analyses locally at each participating registry
- submission of the results of analyses, by each participating center, to the data coordinating center
- combination and aggregation of the results, by the data coordinating center, and electronic distribution of study-wide results to all participants.
Figure 2 shows an example of successful collaboration among ICOR partners that completed some of these requirements; this example can serve as a pilot for future similar projects8.
Stage III: Stratified Data Sharing
In addition to the process described for Stage II, this method includes hierarchical tabulation of the summaries of the data. As a result of stratification, the data are shared at a group level and the smallest “cell” should be large enough to not allow identification of the individuals. One can use the data from each registry to plot the summaries of data from different data sources by specific populations and devices9. It is critical to create the patient strata on the basis of cross-classification of the main factors such as age, sex, diagnosis, important comorbidities, and device characteristics. The larger the number of cross-classification variables, the smaller the cells and more resource-extensive the process for data assembly for collaborative work. This method has some limitations when one is analyzing survival data such as revision surgery occurring over time. In that instance, the stratified data based on cross-classification must be created for each time period relevant for evaluation. For example, if ten-year survival is considered for analysis of implant performance and a one-year interval is considered a reasonable time period for data reporting, then the stratified data need to be available for all ten periods of reporting.
This method allows calculation of cumulative age, sex, or any other cross-classification-specific implant revision rate and comparisons by exposure categories. Furthermore, it allows robust risk adjustment (based on cross-classification variables) and evaluation of incompleteness of exposure (i.e., device characteristics) by patient characteristics. For example, one can evaluate the impact of various cross-classification variables on exposure missingness using a logistic regression model in which the outcome is the log-odds of the probability of missing exposure as a function of subject characteristics and their interactions. Using this data structure, researchers can also differentiate between uncertainty and variability in exposure (i.e., device characteristics) effects. The larger the number of subjects with a specific device exposure, the smaller the standard error and the uncertainty. One can build multilevel models and calculate average predictive comparisons among the exposure categories (as in the hypothetical example of a hip bearing surface) and estimate the expected predictive difference in the risk of outcome (i.e., revision surgery) for each exposure category, holding stratification variables (age, sex, comorbidities, etc.) constant (Fig. 3).
Stage IV: Defining a Minimum Data Set and Centralized Data Sharing and Analyses
This is the highest level of collaboration among the registries and involves describing each registry's history, data quality, description of included patients (flow chart of included and excluded patients), follow-up achievement, patient demographics, surgical techniques, implants, cumulative survival rates, and reasons for implant revisions (e.g., all causes, aseptic revision, and revision due to infection).
The critical issues are the agreement to share data and the availability of a minimum set of variables within each registry as well as the harmonization of data definitions for these variables. Even within the Nordic Arthroplasty Register Association (NARA), where countries are closely related and have many variables in common (i.e., age, date of surgery, date of revision, date of death, sex, and laterality), the countries substantially differ with regard to patient selection, choices of implants and fixation methods, data collection methods, diagnoses, and lists of variables, making these registries not compatible10.
There are also many regulatory hurdles to overcome before data sharing is possible. In the ICOR participating countries, each patient had a personal identifier such as a Social Security number or national civil registration number. All data need to be de-identified, and it must be ensured that re-identification is not possible. For example, within NARA registries, each patient and hospital were assigned a unique number in the newly created combined database and a specific code was given for nationality. This is a time-consuming and exhaustive process and needs to be implemented within each national registry. The researchers need to ensure the confidentiality of data, closely following all regulations in each participating country. This process also requires substantial legal support, which can increase the cost burden and hinder the start of research initiatives.
Possible Surveillance Projects Within ICOR
To implement public health surveillance, ICOR needs to design and implement a dynamic bidirectional data-sharing network among the collaborative partners. The information technology (IT) infrastructure is important to create a data-sharing network that will ensure strict data security, protect personal health information privacy, and automate distributed queries of the harmonized data sets11. Within such a network, the coordination of the exchange of clinical data and analytic results among the study participants will be conducted through the development and implementation of a secure clinical data-sharing and query system, based on an active surveillance framework. The integrated network will be composed of a secure electronic network coordinating the exchange of summary-level information between each of the participating orthopaedic registries and the central data coordinating center. The orthopaedic registry data can be integrated with use of local systems implemented at each orthopaedic registry site.
Aside from the development of a common data model for specific studies as noted above, this method requires installation of advanced software at each participating site. While this method does not require combining the data sets, registries have legitimate concerns related to installation of the software in their registry data system or loading of the data in an environment that allows automated querying of their data. There are also challenges in implementing software systems into various IT environments, such as server-space limitations, compatibility and scalability issues, security and regulatory requirements, long-term system maintenance, and identification of technical resource requirements.
ICOR will require substantial data harmonization for any level of collaboration. Regulatory focus for many questions will ensure adoption of a standard classification of device data, a critical step for the meaningful collaborative investigation of devices. To conduct successful multinational research, ICOR will have a number of tools available to facilitate collaboration at various stages. The staged approach will ensure that the consortium is productive and will address the stakeholders’ needs even when registries request more time for highest-level partnership. Among the tools available to ICOR, decentralized approaches (Stages I, II, and III) will ensure fast and efficient collaboration with little or no loss of quality.