"Hasty publications in prominent medical journals appear to provide the data used by doctors to recommend treatment." These words were voiced by the cab driver transporting me to The Journal offices. The cab driver remarked that he had driven me in the past and remembered that I was a doctor. He informed me that he was wary of doctors because the treatment that they prescribed was usually based on inadequate and incomplete data. The driver proceeded to cite specific examples of publications that, one year, would tout a particular association as a causal relationship and then, in a subsequent year, would publish another study, often by the same authors, that completely refuted the original conclusions. The driver went on to observe that, in the haste to get information into print and the compulsion to provide the sound bite of the day, authors and journals often disregard scientific methodology and long-term follow-up.
This conversation points to the sophistication of the public and our patients in relation to the medical sciences. This interest is probably the reason that articles in leading newspapers, magazines, and journals caution us to avoid drawing hasty conclusions based on inadequate data. Devotees of technology, manufacturers, consumers, those with political and personal agendas, medical societies, advocates of alternative medicine, physicians, non-physician caregivers, the media, even peer-reviewed medical journals, and "our not so gently graying baby boomers"11 are sending mixed signals in their efforts to hold onto a piece of the health-care pie.
The baby boomers, reared to believe that they can retain their youthful bodies, are ably supported in this endeavor by clinicians, often belonging to the same peer group, who believe that they can reconstruct any part of the body. This generation expects to maintain the status quo—that is, to maintain the same level of physical activity—regardless of age and extent of disease. This belief appears to be ubiquitous. An article7 in The New York Times magazine section on September 6, 1998, highlights the problem of inadequate and insufficient scientific studies on the effectiveness and long-term consequences of various operations. The article discusses how the knee has replaced the back in terms of being the most frequent site of musculoskeletal problems that prompt individuals to seek medical care. The knee has become a growth industry and, in the words of the author, Cynthia Gorney: "The flourishing specialty of sports medicine relies on knees … for its own robust health." To emphasize this almost morbid interest (albeit for diverse reasons), the article states that forty-seven of sixty papers presented at the Annual Meeting of the American Orthopaedic Society for Sports Medicine, held in Vancouver, British Columbia, in July 1998, addressed some aspect of injury to the knee.
The author of that article, a forty-two-year-old woman, was informed that even after a reconstructive procedure on the knee she could not play soccer with her team, go downhill skiing, or continue distance running. In fact, the surgeon recommended a generalized reduction in physical activity. The patient elected to have the operation as she wanted to participate in sports for as long as she "could get away with it." It is to be hoped that there was some discussion about the possibility of early osteoarthrosis and the limited options available to a young individual. The article also mentions a seventeen-year-old girl who was having a second reconstructive procedure on the knee so that she could return to sports. It seems as though everyone concerned is engaged in magical thinking (believing in a fountain of youth) and prefers to avoid reflecting on possible long-term adverse effects. The issue is how high you can jump, how much farther you can run, and how much longer you can continue with intensive sports activities, disregarding the natural history of disease. Children and adolescents participate in a plethora of sports simultaneously and at a highly competitive level, often at the urging of their family and sports coach. Once again, magical thinking rules the day and generates a belief that these individuals, with this drive, are the select few to whom nothing bad can happen. These beliefs are further fueled by the publication of studies consisting of observational data based on small groups of patients in a variety of magazines, non-peer-reviewed journals, and even well respected journals.
For example, a publication in a leading journal2, in 1994, reported the results of autologous chondrocyte transplantation in sixteen patients, ranging in age from fourteen to forty-eight years, who had a lesion on the femoral condyle that was 1.6 to 5.0 square centimeters in size. The patients were followed for one year and four months to four years and eleven months. The results were graded as excellent in six patients, good in eight, and poor in two. The paper does not provide sufficient detail about the preoperative findings, the level of activity, the course of the symptoms, the previous treatment, or the specific indication for the procedure. There is no definition of what constitutes strenuous activity. It would be difficult to extrapolate the findings and recommend this procedure on the basis of the results in this small group of patients with a wide range of ages. This article generated enormous pressure—from the corporate world, surgeons, and spokespersons—on third-party payers to cover the costs of this procedure. The pressure led to a rapidly put-together symposium at a meeting of the American Academy of Orthopaedic Surgeons, where the procedure was discussed in the context of other treatment options. It was not given blanket approval. Instead, the procedure was considered to have merit but to require further study.
In another recent article on chrondocyte transplantation5, published in The Wall Street Journal on July 27, 1998, one orthopaedist commented that "this was marketed before a lot of the science and clinical studies were done" and another asked: "To what do you attribute the improvement?"—that is, to the cells or the periosteum or to associated procedures? Interestingly, research laboratories in the United States, England, and several European countries grow the chondrocytes at a fraction of the charges billed by commercial enterprises. Although the procedure was popularized more than fifteen years ago in Sweden, it is only now that a randomized study with controls is being performed to determine if it is the periosteum or the cells that cause the cartilage defect to heal. To date, no articles on the clinical results of this method have passed the vigorous peer-review standards of The Journal of Bone and Joint Surgery.
The dedication of an editor to the maintenance of scientific accuracy is supported by the recent article in The New York Times by Jane Brody3. "A Study Guide to Scientific Studies," published on August 11, 1998, discusses three methods of performing scientific studies. One method is the laboratory experiment, which can be tightly controlled; however, the conclusions cannot always be extrapolated to humans. Another method is epidemiological research, in which observational studies are used. Such studies cannot establish cause and effect; they can only suggest a relationship. They are best at identifying powerful associations, and the larger the study and the longer the follow-up period the more certain one can be of the results. The third type of research is the clinical trial. Brody suggested that a randomized clinical trial needs to be performed in order to confirm that estrogen replacement is responsible for a lower risk of heart disease.
Brody must be clairvoyant since an Editorial10 on the same topic appeared in The Journal of the American Medical Association on August 19, 1998. The Editorial was a response to the results of the Heart and Estrogen/Progestin Replacement Study, published in an article8 in the same issue. This randomized, controlled trial of 2763 postmenopausal women who had established coronary disease showed that treatment with estrogen and progestin did not reduce the rate of coronary-heart-disease events, such as non-fatal myocardial infarction, or the rate of death. In the Editorial10, Diana Petitti, M.D., commented on the need for a "commitment to randomized trials as the standard of proof," especially "when the public health implications are so great." Petitti observed that the findings of the Heart and Estrogen/Progestin Replacement Study are "a sobering reminder of the limitations of observational research, the incompleteness of current understanding of the mechanisms of vascular disease, and the dangers of extrapolation." The question that should be asked is whether all of the parties concerned—the authors and the commercial interests—will present this new information on estrogen-progestin replacement therapy to the clinicians, the health-care providers, and the public with the same fanfare as that accompanying the initial push for the use of such therapy.
The use of Ginkgo biloba tripled after publication of an article in The Journal of the American Medical Association9 on October 22, 1997. That article concluded that the extract appears "capable of stabilizing and, in a substantial number of cases, improving the cognitive performance and the social functioning of demented persons for 6 months to 1 year." The resulting media blitz suggested that the extract from leaves may in fact be an all-purpose brain-booster. There are several problems with these conclusions. Only 202 of the 327 patients were available for the end-point analysis. Also, a comment made in the article suggests that improvements in cognition and social behaviors, although statistically significant, were not sufficiently large to be clinically meaningful. Unfortunately, this caveat was not included in the abstract. In these days of sound bites, it is more than likely, given the pressures of time, that only the abstract will be read by a large number of people.
Dutch researchers found only eight reasonably sound clinical trials of Ginkgo biloba for patients who have cerebral insufficiency6. Jos Kleijnen, one of these researchers, commented that recent, as yet unpublished, studies in Europe in which placebos that had the same bitter taste as ginkgo were used showed that ginkgo had no benefit after three to six months of use. In a review of the literature, medical researchers at Consumer Reports on Health6 found only two studies of the ability of the extract to boost memory in healthy people; in one, performed on eight young women who took various doses of the extract or a placebo, no effect was seen until the dose reached 600 milligrams. In the second study, on twelve volunteers, even a high dose had no effect.
In the words of Brody, "… it often takes many studies that examine an issue from many angles" before one can arrive at conclusions that can be used to influence or alter medical treatment3. Hasty publications can scare the public, generate enormous costs for the health-care system, and produce unwanted repercussions affecting all segments of society. In a second article4, published on August 18, 1998, Brody discussed a report published in 1981 linking pancreatic cancer to the consumption of coffee. The study was quoted as stating that two cups of coffee a day doubled the risk and five cups tripled the risk. This finding was not confirmed in any other study, and five years later the Harvard group repeated the study and failed to confirm their initial results.
The Journal of Bone and Joint Surgery insists on accurate scientific methodology, checks all statements including references, and makes certain that the conclusions and recommendations are supported by the data. This process generates questions about the problem being studied and its relevance; the appropriateness of the methodology that was used to answer the question; the inclusion and exclusion criteria; the uniformity of the group selected; and, in the case of comparison studies, the comparability of the study and control groups. The Journal then checks for accuracy of numbers, incomplete data, contradictory statements, accuracy of data derived from citations, and consistency among the numbers in the Abstract, Materials and Methods, Results, and Discussion. In order to maintain scientific accuracy, avoid ambiguity, and prevent misleading conclusions, it is necessary to check each number and word in a manuscript. The readers would receive an educational experience if The Journal were to publish an article in its initial form, each set of corrections and questions raised, the final version sent for copy-editing, the galley proofs with the additional questions generated during copy-editing, and the final printed paper. However, it is unlikely that anyone would agree to have his or her initial manuscript published along with the final version.
The media, in the form of magazines and newspapers, has taken on the task of monitoring and assessing the scientific accuracy and validity of published data. This should be our role. We should not sacrifice excellence and good scientific methodology in a desire to please vested interests. We must avoid the behavior of politicians who are constantly succumbing to the pressure of various lobbyists. Science is the patient accumulation and verification of facts. We should not deviate from this standard if we are to provide the best medical care. In the words of Jacques Barzun, in his book The House of Intellect, "What matters to a nation is whether the best product, or in certain cases the high average, which prides itself on excellence, deserves its reputation."
Leela Rangaswamy, M.D.