Abstract
Abstract:
Recent events have made us question our ability to monitor the introduction and performance of a new prosthetic joint replacement. These concerns are mirrored in many other countries, even those with joint registries and other systems for detecting poorly performing implants. In the United Kingdom, as is the case in most European Union countries, we have assessment processes in place, particularly the CE (Conformité Européenne) mark, the National Joint Registry, and the Orthopaedic Data Evaluation Panel. It is important to realize that these organizations can only react to poor performance; it is not within their power as monitoring agencies to prevent a poor implant from entering the market. When an implant has been shown to perform badly, it is referred to the Medicines and Healthcare products Regulatory Agency (MHRA), which then discusses the report with the manufacturer. When it is deemed that there is unequivocal evidence of a serious problem with the implant, the MHRA issues alerts to all hospitals and practitioners. We have continuing concerns about how to assess and monitor modifications to already existing devices. Suggestions are made for improvements to this system, and the authors will welcome feedback.
At present, for an implant company to introduce a new device to be used in the European Union (EU) the company must satisfy one of the notified bodies of the worthiness of the device. There are eighty-three notified bodies in the EU, spread throughout the member countries. Once the notified body has established the safety of the device, a CE (Conformité Européenne) mark is awarded and the device can then be marketed throughout the EU without hindrance.
To obtain a CE mark, the manufacturer must produce the following evidence to match the rigorous definition of a class-3 device1,2:
- Mechanical and biological testing (this may be based on a literature search or previous reports) to show compliance with international standards
- Strong evidence that the device that the company hopes to market is sufficiently similar to a device with a good track record. Clinical trial data (optional, and definition of standards not clearly defined)
- Agreement to report all failures to the competent authority (the Medicines and Healthcare products Regulatory Agency [MHRA] in the United Kingdom [U.K.])
The process of monitoring begins with the manufacturer formally notifying the following agencies of the existence of their new implant:
- The National Joint Registry (NJR) (for hip, knee, ankle, and shoulder devices)
- The Orthopaedic Data Evaluation Panel (ODEP) (at present, only for hip devices)
The NJR contractor (at present, Northgate Solutions) enters the details of the device on the main database, after which complying hospitals can proceed to upload data.
The NJR data are trawled twice a year by the NJR contractor, and a linkage between primary and revision procedures on each implant is sought. Where a linkage has been established and the revision has been reported, those data are assessed against the performance of the same implant brand both in terms of volume and time. This generates the PTIR (patient-time incident rate). The PTIR is the rate of occurrences of an incident (i.e., revision) for a given patient time (usually reported as revisions per 100 observed component years). Thus, if a prosthesis has accumulated 1000 patient years and there have been three revisions, the PTIR will be 0.3 (Table I).
The PTIR for the identified implant is compared with the average PTIR for that particular group of implants (e.g., cemented cup, surface replacement, or unicondylar knee). When the PTIR of the cohort of that implant is found to be more than twice the group average, the implant is defined as a potential outlier. PTIR data have been reported to the NJR Implant Performance Committee by the contractor (Northgate) since 2010.
A high level of confidentiality is maintained throughout this process. When the potential outlier prosthesis is found to have been used in more than 100 cases, the Scrutiny Group of the NJR Implant Performance Committee processes the data and issues a red alert to MHRA. The verification process includes further statistical analysis and basic “sanity” checks to ensure that the computer has identified valid data.
When the potential outlier prosthesis is found to have been used in fewer than 100 patients, the company is contacted, the MHRA is notified, and an amber alert is issued. The manufacturers of implants showing a trend toward outlier status are informed of the situation.
In addition to PTIR, other triggers can initiate an analysis of a particular implant performance. These triggers can include contact by clinicians, scientific presentations (oral or written), funnel plots, and associations with other similar devices.
The Orthopaedic Data Evaluation Panel
The ODEP functions differently than the NJR does, although there are strong links between the two organizations. The ODEP offers the manufacturer the opportunity of presenting data to demonstrate the implant's efficacy. The details of this process are fully presented on the National Health Service (NHS) Supply Chain web site3.
Companies submit the details regarding their implant (pre-entry) and agree that they must be rigorous with post-market surveillance. All of their implants must be registered with the NJR. The companies will then be expected to produce evidence that they are continuing to meet the 3.5-year and seven-year benchmarks until they achieve the ten-year benchmark. Each benchmark is a defined standard and for it to be achieved the necessary information has to be presented. This information will include the details of the implant, details of trials or presentations, any international joint registry data, cohort sizes, revision numbers, Kaplan-Meier survival data, and mortality figures (Table II). Failure to adequately progress through the benchmarks leads to de-recognition of the device. It is anticipated that the length of surveillance will be expanded to fifteen years, and discussion is ongoing as to whether knee implants should be included. Implant companies often use ODEP ratings in their publicity, as the ODEP is known as an agency that promotes the good.
The MHRA will investigate adverse reports and produce safety warnings and alerts when the need is indicated. Before coming to any decision, they ask the company to talk to them about the potentially outlier data that have been produced by the NJR. The MHRA is staffed with highly qualified scientists who have access to other independent specialist advice when necessary.
Our system is relatively immature, and we have been guilty of suggesting that a particular device is an outlier only to find that, when the data have been reevaluated, the product has performed satisfactorily.
It is essential that poor performance of an implant is detected early, and it is a heavy responsibility to identify a potential outlier. The consequences for the manufacturing company are obviously very far reaching.
In order to maintain the reputation and integrity of the registry, while the protection of patients is paramount, the accuracy and confidence level of the data are also essential.
With all the ASR implants (DePuy Orthopaedics, Warsaw, Indiana), including the ASR acetabular cup for hip resurfacing arthroplasty or total hip replacement, the ASR surface replacement head for hip resurfacing arthroplasty, and the ASR XL femoral head for total hip replacement, the problems of detection were several. The ASR implant came out at the same time that NJR was started, and our first record of ASR prostheses being implanted was in late 2003. At that time, compliance was approximately 74% and consent was only 69%. Compliance is the percentage of registrations of procedures with NJR against the number of implants sold. “Consent” describes the number of patients who have given their written consent to the storing and use of their data as a percentage against compliance. This meant that, at the time, the linkability was <50%, statistically leading to wide confidence levels. (Compliance, consent, and linkability are now in the order of 99%, 84%, and 94%, respectively.) Linkability is best achieved by using the unique NHS numbers that are assigned to all patients in the U.K. Most countries have a numeric identifier of one type or another for all of their citizens.
The number of procedures in which the ASR femoral head or stem were used (as identified from the NJR database) during the years 2003 through 2010 is shown in Tables III and IV.
The graphs shown in Figures 1 and 2 show that there is a discrepancy between the number of acetabular cups and femoral heads that were used during this time period; this discrepancy can be explained by the fact that 50% of the cups were used with a large metal-on-metal femoral head with a conventional stem. This reduced the size of some of the involved cohorts and made for more difficult detection of outliers. The poorly performing, large metal-on-metal heads attached to conventional femoral stems were hidden within the conventional stem (usually uncemented) database. Their poor performance was diluted by the good performance of the conventional stems, and they were initially overlooked.
The NJR did not confidently determine that there was a potential problem with the ASR implant until the 2008 data were analyzed. The NJR reported the ASR implant to the MHRA in April 2010, at which time an MHRA device alert was issued.
Data critical of the performance of the ASR were presented at numerous scientific meetings in 2008 and onward to the time of this writing, particularly at meetings of the British Hip Society (BHS) and the British Orthopaedic Association (BOA). This factor, rather than the MHRA alert in July 2010, almost certainly led to the subsequent drop in sales of that implant.
The Australian Registry picked out the ASR implant as being a potential outlier before the NJR did, and this may well be due to the fact that the Australian Joint Registry has been in existence longer and had 100% compliance with regard to the ASR implant.
The ASR implant was withdrawn from the market in August 2010.
To some extent, it has been in the wake of the ASR implant that we have adopted the PTIR system as it is more sensitive in picking up outliers. At the time that the ASR implant was running into difficulties, we were using funnel plots.
In a recent trawl of the NJR database for potential implant outliers, when using the rule that a number more than twice the PTIR for that group would spur an investigation, twenty-four potential red alerts and forty-one amber alerts were detected for ongoing analysis. There was only one red alert identified with use of the funnel-plot technique.
It could be asserted that we are now becoming too aggressive in trying to detect potential outliers; however, the data are always carefully analyzed by a team that includes patient's representatives, clinicians, scientists, and statisticians. Brand confidentiality is maintained throughout the process. We believe that a hypersensitive state is preferable to a somnolent one.
Until recently, the NJR was voluntary and we did not have as much confidence about the data as we do now. Unless compliance is well over 95%, there will never be confidence in data from any registry. Compliance means not only registering the index primary operation but the revision operation as well. A subtle omission of one or two uploads for revision in a small cohort could make a huge difference in outlier detection. Because of our concerns about missing revisions, we have also used hospital episode statistics (HES) data. Every procedure undertaken in an NHS hospital is registered as an HES episode. HES do not apply to the private sector, but this drawback is offset by the fact that most revision procedures are undertaken in NHS hospitals. Unfortunately, and rather later than we would have wished, we realized that some HES data were inaccurate, resulting in potentially overestimated revisions. An example of an inaccuracy was coding for revision when, in fact, only an irrigation was performed.
In England and Wales, reporting to the NJR is now mandatory and our advice to anyone setting up a new registry is to ensure that all data are collected and that the linkability between primary and revision surgery is secure.
There can be no doubt that the increasing maturity of a registry leads to increased confidence in its output. When the NJR started, we were all learning our way. We had the Scandinavian registries to guide us, but it has taken a while for us to assimilate data and report with the confidence that we now feel is justified.
With the continuing awareness of potential large-head metal-on-metal problems as well as the large number of patients who may be involved, the MHRA together with the BOA and the BHS worked with the NJR to assess the extent of the problem and to develop guidance4. It had taken only a matter of a few hours for Northgate to identify all patients who had received ASR implants and the hospitals where they had been seen. This metal-on-metal committee continues to review the information as it comes in and reviews and updates the guidance when necessary.
It has always been emphasized that the great majority of metal-on-metal surface replacement operations that have been undertaken during the past ten years or more have produced acceptable results, with high patient satisfaction. According to the NJR data, the Birmingham surface replacement (Smith & Nephew, London, United Kingdom) is the market leader, has a 10A ODEP rating5, and has 97% survivorship at five years. The early poor results with resurfacing devices were usually due to operative technical failure or patient selection. The national joint registries have all reported inferior results of surface replacement in women who are older than fifty years of age and who have small hips6.
The guidance, referred to above, offered jointly by the MHRA, BOA, and BHS Metal-on-Metal Committee has emphasized that metal-on-metal surface replacement should only be performed on appropriate patients, that those patients should receive annual follow-up, and that the serum cobalt and serum chromium levels in those patients should be measured7. Patients in whom either level is above 7 parts per billion require further investigation and usually the use of axial imaging, such as ultrasound or metal artifact removal sequence magnetic resonance imaging (MARS MRI) scans. Any patient with symptoms (such as groin pain) should undergo full investigation with advanced imaging techniques.
The U.K., through the British Orthopaedic Association and the MHRA, is developing new guidelines for the introduction of new prostheses and the modification of existing devices. It is hoped that with the development of the International Consortium of Orthopaedic Registries (ICOR) and other agencies, there can be some international agreement with regard to these guidelines. We are of the opinion that a step-by-step approach should be followed and that a committee should be formed to oversee the introduction and performance of new implants and the modifications of existing ones. Guidelines should be introduced that reflect the novelty of designs and the degree of incumbent risk.
While safeguarding patients is paramount, it is also important that responsible research and development in the field of joint replacement continues. Our evidence suggests that so-called “minor modifications” to the design of a prosthesis may be much less innocent than could have been expected in some cases. Thus, post-market surveillance of new devices and modifications to existing devices should be undertaken for at least ten years. The intensity of the surveillance should depend on the novelty of the new design. It is envisaged that all joint registries will be monitoring individual prostheses indefinitely or at least until all of the patients who received that particular prosthesis are dead.
We feel that it is essential that all of the world's joint registries should communicate with one another when a potential outlier implant is identified, particularly when the cohorts are small and the implants are at an early stage of study and statistical analysis is limited.
Eur-Lex 31993L0042. Council Directive 93/42/EEC of 14 June 1993 concerning medical devices. .
EUR-Lex 32005L0050. Commission Directive 2005/50/EC of 11 August 2005 on the reclassification of hip, knee and shoulder joint replacements in the framework of Council Directive 93/42/EEC concerning medical devices. .
ODEP. NHS Supply Chain. .
Medical device alert: All metal-on-metal (MoM) hip replacements (MDA/2010/033). London: Medicines and Healthcare Products Regulatory Agency: 2010 Apr 22. .
8th Annual Report of the National Joint Registry for England and Wales. The NJR Centre: Hemel Hempstead, United Kingdom; 2011. .
McBryde
CW;
Theivendran
K;
Thomas
AM;
Treacy
RB;
Pynsent
PB. The influence of head size and sex on the outcome of Birmingham hip resurfacing, J Bone Joint Surg Am.
2010;92:105-12.[PubMed][CrossRef]
Medicines and Healthcare Products Regulatory Agency (MHRA). Medical Device Alert: All metal-on-metal (MoM) hip replacements. .