To the editor:

I read with interest the article entitled "Magnetic resonance imaging of the knee in children and adolescents, its role in clinical decision making", by Scott Luhmann and colleagues (1), both because I practice musculoskeletal radiology and because the work originated from a hospital affiliated with my own. I applaud the authors for incorporating a personal review of magnetic resonance imaging (MRI) studies into their practices. However, I do not believe that their study design supports their statement questioning the necessity and appropriateness of a routine radiologist’s interpretation of knee MRI scans obtained in children and adolescents.

The operating surgeon reached his preoperative diagnosis by synthesizing information from the clinical history, physical examination, radiographs, radiologist’s interpretation of the magnetic resonance imaging scans, and a personal review of the magnetic resonance images. The fact that the accuracy of this final preoperative diagnosis based on a combination these five factors was higher than that of one of its components (the radiologist’s interpretation of the magnetic resonance imaging) is expected. But this finding cannot be used to question this component’s contribution to the overall preoperative diagnosis any more than it could be used to question whether another component – like physical examination – was necessary or appropriate. For example, their Table E-1 shows that the sensitivity and specificity of the radiologists’ magnetic resonance imaging interpretation were 79 – 94% and 87 – 91%, respectively, for meniscal tears. These values fall well within the range reported in large, prospective series of magnetic resonance imaging results in adults (2). The sensitivity and specificity for the surgeon’s final, composite diagnosis of meniscal tear were 94 – 96% and 97%. If the authors’ physical examination of the pediatric knee for meniscal tears were as good as that found in large, prospective, adult series, which report sensitivities 16 – 85% and specificity of 84% (3, 4), then their logic would question the necessity and appropriateness of physical examination even more than it would for having a radiologist interpret the magnetic resonance imaging studies. But these issues could only be addressed in a study that compared the accuracy of diagnoses reached with and without a specific piece of information (like the radiologist’s MRI interpretation or the results of physical examination), which was not the study design employed in the research of Luhmann et al.

Additionally, since the preoperative diagnosis and radiologist’s MRI interpretation were not independent (because the latter was a factor in the former), the use of the kappa statistic to compare the two is not appropriate (5).

Selection bias and observer bias also likely influence the reported results. That no intraarticular disorder was found arthroscopically in only four of 96 knees suggests that many children did not undergo operation after evaluation of their knee symptoms. In these knees, likely one or several of the factors (history, physical examination, radiographs, radiologist’s MRI interpretation, surgeon’s MRI review) influenced the decision not to operate. How large a role the radiologist’s MRI interpretation played in this decision cannot be determined from the study, but it seems premature to question its utility without investigation. Additionally, since the operating surgeon was also the physician who synthesized the preoperative diagnosis, observer bias could not be avoided. Finding that the observer’s preoperative diagnosis was frequently supported by his own arthroscopic findings is expected in this circumstance.

References:

1. Luhmann SJ, Schootman M, Gordon JE, Wright RW. Magnetic resonance imaging of the knee in children and adolescents. J Bone Joint Surg 2005; 87-A;497-502.

2. Rubin DA, Paletta GA Jr. Current concepts and controversies in meniscal imaging. MRI Clinics North Am 2000; 8:243-270.

3. Fowler PJ, Lubliner JA. The predictive value of five clinical signs in the evaluation of meniscal pathology. Arthroscopy 1989; 5:184- 186.

4. O’Shea KJ, Murphy KP, Heekin RD, Herzwurm PJ. The diagnostic accuracy of history, physical examination, and radiographs in the evaluation of traumatic knee disorders. Am J Sports Med 1996; 24:164-167.

5. Cohen J. A coefficient of agreement for nominal scales. Educational Psychological Measurement 1960; 20:37-46.

To the Editor:

I appreciate the letter from Dr. Ehara regarding our publication, “Magnetic Resonance Imaging of the Knee in Children and Adolescents. Its Role in Clinical Decision-Making”. He addresses several issues with the manuscript which I will discuss separately.

1. He raises the issue of method of data collection. As stated in the manuscript the preoperative radiologist MRI interpretation and the surgeon preoperative diagnosis were documented prior to performing the arthroscopic surgery. He points to potential bias in the data collection which may alter the findings of the manuscript. Data analysis was performed by an experienced medical statistician, who obviously is not involved in the care of the patients.

2. This study replicates “real world” clinical medicine. MRI scans ordered by outside physicians at outside imaging centers often have little to no information about the clinical diagnosis (and typically do not have any plain radiographs as these are at another facility), especially those ordered by primary care physicians.

3. The use of “probable” and “possible” was only found on the radiologists formal MRI interpretation. The surgeon’s preoperative diagnosis was always that the problem was or was not present. This issue of how to treat the “probable” and “possible” MRI diagnoses was discussed with the other authors on the manuscript. There was a consensus that we should include these as positive findings since primary care givers and patients treat these as “real” diagnoses, hence they drive patient referrals to the orthopaedic surgeon’s office. It is at that point that the onus of responsibility is on the orthopaedic surgeon to rule out this as a real problem, since primary care givers, families, patients and insurance companies think the MRI is the “gold standard” for definitive knee diagnosis.

4. I would refer Dr. Ehara to the Methods section of the manuscript about this point. All preoperative clinical surgeon diagnoses were recorded after taking a history, performing a physical examination, and reviewing the plain radiographs and the MRI scan (and MRI interpretation), regardless of where the MRI was performed.

Dr. Ehara raises concern about the ability to generalize the findings of this study to clinical medicine. The finding of this study demonstrate the orthopaedic surgeon should personally review all MRI examinaations and not rely upon the radiologists interpretation. Extrapolating the findings of any study beyond the study purposes should always be done with extreme caution.

To the Editor:

I appreciate the response from Dr. Rama regarding our publication, “Magnetic Resonance Imaging of the Knee in Children and Adolescents. Its Role in Clinical Decision-Making”. As previously stated in the manuscript and in an earlier response to a “Letter to the Editor”, the reason for this study was to address, in part, the role of MRI scans in the treatment of adolescent knee problems. The pervasive use of MRI for knee pathology by primary care givers and orthopaedic surgeons warrants examination of the way we utilize this imaging modality. I would speculate that in the United States there are thousands of MRI scans performed each year which are either unnecessary or suboptimally utilized by the requesting physician. Too often the MRI interpretation of the knee pathology is considered as the “gold standard” by primary care givers and orthopaedic surgeons and treatment is then based on this interpretation, regardless of the patient’s symptoms or examination. The physician requesting the MRI scan should personally review the MRI scan and then incorporate this information into the patient’s history and physical examination. Unfortunately, MRI is often inappropriately used a screening modality, prior to referral to an orthopaedic surgeon. The orthopaedic surgeon is in the optimal position to utilize this powerful imaging modality due to their knowledge of the musculoskeletal system derived from their training and continued clinical practice in the office setting and in the operating room suite.

Dr. Rama proposed an “ideal study” from which we may obtain “the intended purpose and find the answers.” His interpretation of the study purposes is only one of many ways in which one may want to perform this type of study. I hope this manuscript encourages Dr. Rama to initiate a study at his institution, as he has outlined, to further evaluate the place of MRI in the treatment of adolescent knee problems.

To the Editor:

In the paper authored by Luhmann and his colleagues in the current issue of the Journal (1), the authors proved that the arthroscopic findings were better correlated with the orthopedic surgeon's diagnosis that was based on clinical examination, plain radiographs and MR imaging when compared to the interpretation of MR imaging by a group of radiologists. Because of methodological flaws, I think that we can hardly extend this conclusion to orthopedic surgeons and radiologists in general. The reasons are as follows:

First, comparison of two groups performance should be performed by a third party, or, at least, an investigator not directly involved in the data collection. In this paper, the orthopedic surgeon involved in the clinical evaluation was the first author of this paper, and the radiologists probably consisted of pediatric radiologists with different expertise in these knee disorders.

Second, the radiologists had access to plain radiographs and the orthopedic surgeon's presumptive diagnosis. It was uncertain whether the results of clinical assessment and the level of confidence in the diagnosis were actually available to the radiologists. Radiologists evaluations of imaging studies are based on trust in the referring physician's clinical assessments, and radiologists can be easily misled if the referring physician's assessment is inadequate or inaccurate. Without close communication and feedback, the radiologists's interpretations lose accuracy, since there are substantial overlaps of normal and borderline abnormalities in many circumstances.

Third, the authors included "probable" and "possible" diagnoses into the positive diagnosis without establishing consensus in the confidence level. The possible diagnosis may have been made only to raise a suspicion even if the possibility was considered to be small.

Fourth, the orthopedic surgeon knew the findings of the MR imaging done outside the institution at the initial assessment in 41 of 96 patients. So, in nearly half of the cases, the clinical assessment actually included MR imaging findings.

I believe that, because of these methodological flaws, the conclusion of this paper should not be extended to clinical practice in the other settings. In addition, the opinion of the radiologists is not included in this paper, and the statements appear one-sided.

Reference

1. Luhmann SJ, Schootman M, Gordon JE, Wright RW. Magnetic resonance imaging of the knee in children and adolescents. J Bone Joint Surg 2005;87A;497-502

To the Editor:

We read the article by Luhmann, et al, with great interest. We found a serious flaw in the study design. The authors have stated that one of the two purposes of their study is ‘to determine the benefit of a personal review of the magnetic resonance (MR) scan of the knee by the orthopaedic surgeon, as a routine part of the diagnostic evaluation’. If that is the purpose, they should have compared the surgeon’s diagnostic accuracy before and after reviewing the MR scan. Instead, they compared the diagnostic accuracy of the radiologist’s report of the MR scan with the diagnostic accuracy of the surgeon, who correlated the clinical findings and the radiologist’s report with his/her personal interpretation of the MR scan. So the results of this study cannot estimate the true benefit of the surgeon’s review of the MR scan and the author’s conclusion regarding this is questionable.

The clinical input of the radiologist is usually limited to a ‘few words’ about the surgeon’s presumptive clinical diagnosis. Moreover his/her report is rather a radiological diagnosis than a true diagnosis. On the other hand, the surgeon is in a far more advantageous position, as he/she has input of both the detailed clinical findings and the radiologist’s report. A greater diagnostic accuracy is expected from the surgeon as he takes the best of the both and correlates them. If one wants to estimate whether this accuracy can be further improved by personal reviewing of the MR scans by the surgeon or not, the ideal study design should be as follows: the diagnostic accuracy of the surgeon should be assessed and compared at three different stages: (1) before ordering an MR scan (2) after studying the radiologist’s report of the MR scan and correlating it with the clinical findings (3) after evaluating the MR scans personally and correlating with the previous findings. Only this sort of study design will serve the intended purpose and find the answers.

To the Editor:

I appreciate the response from Dr. Chow regarding our publication, “Magnetic Resonance Imaging of the Knee in Children and Adolescents. Its Role in Clinical Decision-Making”. The reason we performed this study was because of my experience caring for pediatric and adolescent knee injuries in our Sports Medicine Clinic at St. Louis Children’s Hospital.

On average there are two to three patients a month referred to our clinic with incorrect diagnoses of knee injuries that were based solely on the formal radiologic interpretation of an MRI. The design of this study was to purposely include MRI scans from our hospital and from MRI centers outside of our institution. A study constructed in such a fashion would make our findings more generalizable and applicable to the general orthopaedic surgeon.

I would agree with Dr. Chow that is likely that musculoskeletal fellowship-trained radiologists would be more accurate interpreting pediatric and adolescent knee MRI scans when compared to radiologists without subspecialty training, but to the best of my knowledge such a study has never been published. Since the majority of MRIs in the United States are not interpreted by a musculoskeletal fellowship-trained radiologists, constructing a study as recommended by Dr. Chow would severely limit the clinical relevance to the orthopaedic surgeons who are not at academic medical centers, and do not have access to musculoskeletal fellowship- trained radiologists. It is this group of orthopaedic surgeons who are the ones taking care of the majority of these patients in the United States. Making our study relevant to the majority of orthopaedic surgeons has the greatest likelihood of improving the delivery of orthopaedic care for this group of pediatric and adolescent patients.

To the Editor:

I have serious objections to the manner the study methods were performed as well as the conclusions drawn by the authors. The study conclusions state that there is relatively little or no added value in the radiologists' interpretation of knee MRIs in children and adolescents. However, in the materials and methods section, the interpretations were performed by pediatric fellowship-trained radiologists. This study design is flawed for this reason alone since the interpretations were not conducted by musculoskeletal fellowship-trained radiologists. Therefore, the authors should retract the statement regarding radiologists' interpretations as unnecessary and of no added value.

Musculoskeletal radiologists are trained in orthopedic and sports medicine imaging and have great understanding regarding the fundamentals of orthopaedics, including mechanisms of injury. When working closely with our orthopaedic surgeons/sports medicine colleagues, we have very high positive correlation to true pathology confirmed by arthroscopy or open surgery. The methods applied in this study would be akin to asking pediatric surgeon (or any other fellowship-trained surgeon other than orthopaedics) to diagnose and treat orthopeadic ailments such as ACL or meniscal tears. That would also lead to diagnostic inaccuracies and delay in appropriate treatment for such individuals.

Bernard Chow, MD Director of Musculoskeletal Radiology Santa Barbara Cottage Hospital P.O. Box 689 Santa Barbara, California 93102-0689 bchowmd@cox.net