Although the radical concept of understanding the effect of medical and operative treatment on patient function emanated from Codman3 more than sixty years ago, the orthopaedic community has been actively engaged in the outcomes movement for only the last twelve to fifteen years. Under the leadership of such individuals as Gartland and Sledge, we collectively became aware of the deficiencies apparent in our literature, which had resulted in a less-than-solid foundation on which to make treatment recommendations to patients4,5,9. The technique of small-area analysis pioneered by Wennberg and others revealed the effects of this lack of a solid foundation10,11. Our orthopaedic community is now actively involved in remedying this problem. For eight years, the Outcomes Committee of the American Academy of Orthopaedic Surgeons has provided annual courses on the techniques used in the field of outcomes assessment. We have devoted substantial resources toward providing validated questionnaires and the software to use them. We have done much of the political groundwork to establish broad acceptance for a unified approach to the collection and use of data. We have come a long way indeed.
Despite these accomplishments, two questions remain: how should these data be used, and who should be involved in collecting them? The resources involved in obtaining data are substantial; however, health-care payers have little interest in funding these activities. Therefore, we believe that a timely review of the current status of data collection, future opportunities, and the challenges to be overcome is appropriate.
We begin with a review of the basic principles involved in the selection of outcomes-measurement instruments and then offer an overview of the American Academy of Orthopaedic Surgeons' involvement in providing the critical resources for the orthopaedic community. Next follows a thoughtful review of the Maine experience in the collection of community-practice-based outcomes data; this may be the model for success that we should be striving to imitate at the state and specialty-society levels. Finally, we demonstrate how outcomes data can be linked to financial data in order to provide useful information at the practice level.
Practitioners must clearly understand the separate and distinct issues involved in the assessment of clinical and functional outcomes. Clinical outcomes are what we have traditionally measured in orthopaedic surgery—rates of infection, number of dislocations, range of motion, strength, gait, and so on. These measurements remain important as distinct measures of the success of orthopaedic interventions, and they should be accompanied by an assessment of functional outcomes. These data are obtained from the patient through the use of validated questionnaires, and they focus on the function of the individual as a whole, from the individual's own point of view. All areas of human function, such as physical function, pain, health status, work activity, and activities of daily living, are covered in this type of assessment. The questionnaires are developed with established psychometric principles. The steps are item development, format selection, item reduction, and validation.
Items, or questions, are obtained from experts in the treatment of the entity to be measured, from other questionnaires or, ideally, from patients who have the problem or problems to be assessed. The questionnaire can have a yes-or-no (or endorsable) format, a scaled format (a 3 or 5-point Likert scale with end-point statements), or a visual-analog-scale format (the patient makes a mark on a line of set length). There are distinct advantages and disadvantages to each of these formats. Item reduction involves systematically decreasing the number of items on the basis of the frequency with which the issue is raised by the patients being studied and the relative impact of the item on function. Statistical techniques are used to provide a broad range of items with varied impacts on quality of life and function. Validation encompasses face validity, construct validity, and criterion validity, and it involves reliability and responsiveness testing.
Face validity simply answers the question of whether the questionnaire appears reasonable for the condition or conditions that it was designed to study. Construct validity involves collection of data, development of hypotheses about how the responses to the questionnaire should look, and then assessment of whether the responses are consistent with these hypotheses. Criterion validity involves collection of simultaneous data with the use of so-called gold-standard measures to determine how data collected with the new questionnaire relate to the standard data. Reliability is assessed with a test-retest exercise. The responses to questions should be stable when the patient's clinical status has not changed. Finally, responsiveness is an exercise to test the questionnaire's ability to measure changes in the patient's functional status.
Simply stated, the questionnaire must make sense to patients and practitioners, and it must be proved that it measures what it was designed to measure in a predictable fashion. Surgeons alone generally designed the scales and measures developed previously within the orthopaedic community for application to retrospective case series. These measures lacked patient input, mixed clinical and functional outcomes, were arbitrarily weighted, and were not validated.
The choice of which validated questionnaire to use should center on the purpose of the data collection, the resources available, and practicality issues. Externally funded research projects can use longer questionnaires that are administered by the interviewer. Office or community-based outcomes projects that are funded by the individual practices must use short, less resource-intense instruments.
Examples of validated questionnaires that are useful for musculoskeletal evaluation include the Short Form-36 (SF-36), the Western Ontario and McMaster University Osteoarthritis Index (WOMAC [for arthritis of the lower extremities]), the Sickness Impact Profile (SIP [for conditions that have a more severe functional impact]), the Musculoskeletal Function Assessment (MFA), the Short Musculoskeletal Functional Assessment (SMFA), the Brigham Carpal Tunnel Instrument, and the Musculoskeletal Outcomes Data Evaluation and Management System (MODEMS) questionnaires. All of these questionnaires have been scientifically validated, and some (such as the SF-36 and the SIP) are designed to be more broadly applied across multiple health conditions. Some (such as the WOMAC [for arthritis of the lower extremities] and the Brigham Carpal Tunnel Instrument) are focused on an individual condition, and others (such as the MFA, SMFA, and MODEMS instruments) are designed to focus on musculoskeletal conditions. There is some logic in continuing to develop and validate condition-specific outcomes questionnaires, but only for extremely common conditions. Developing and validating new questionnaires is time-consuming and expensive. Most of our efforts should be concentrated on setting the clinical priorities for study and on developing the mechanisms to cooperatively conduct these studies at the practice, state, and subspecialty-society levels.
Address for Dr. Swiontkowski: Department of Orthopaedic Surgery, University of Minnesota, 420 Delaware Street S.E., Box 492, Minneapolis, Minnesota 55455. E-mail address: swion001@tc.umn.edu.
Critical evaluation of the results of treatment is one of the most important and most effective methods of improving medical care. Valid, reliable information concerning the results or outcomes of treatment makes it possible to identify the most effective treatments and to make informed decisions concerning the allocation of health-care resources. Over the last five years, the American Academy of Orthopaedic Surgeons, the specialty societies that form the Council of Musculoskeletal Specialty Societies (COMSS), and the Council of Spine Societies (COSS) have committed substantial resources to the development of both validated outcomes questionnaires and a national database of information collected from those questionnaires. The goal of this effort is to improve the care of patients who have musculoskeletal disorders by critically analyzing the outcomes of medical and operative treatment with use of questionnaires that are valid, reliable, and sensitive to changes in musculoskeletal function.
In 1993, the American Academy of Orthopaedic Surgeons and many of the member societies of the Council of Musculoskeletal Specialty Societies first came together to pursue this goal. Since then, the project has proceeded through five successive and overlapping stages. Beginning in 1993, outcomes instruments were developed and refined. In 1995, a pilot data-collection project was begun. In 1996, a national database was established and physicians practicing in the United States were given the opportunity to use and contribute to the database—the Musculoskeletal Outcomes Data Evaluation and Management System (MODEMS). The initial questionnaires were then revised, and additional research modules were developed on the basis of the questionnaires; this work was completed in 1997. In 1998, the Musculoskeletal Education and Research Institute (MERI) was established to oversee MODEMS and to promote outcomes research.
Individuals and organizations had developed a variety of outcomes questionnaires before 1993. Some of these questionnaires had been used extensively, but the Academy and the specialty societies recognized that providing a system for analysis of outcomes in patients who had conditions that ranged from fractures to congenital deformities to chronic back pain would require reorganization of available questionnaires and creation of selected new questionnaires. After evaluating the available questionnaires and considering a variety of approaches to assessing the outcomes of treatment in patients who had musculoskeletal disorders, the Academy and the specialty societies established a framework for the creation of four questionnaires that would cover the spectrum of musculoskeletal disorders: (1) pediatric diseases and injuries, (2) diseases and injuries of the spine in adults, (3) diseases and injuries of the upper limb in adults, and (4) diseases and injuries of the lower limb in adults. The Academy and the specialty societies agreed that the questionnaires should be based primarily on symptoms, function, and quality of life and that they should be reliable, validated, and sensitive to changes in musculoskeletal function. In many instances, the four instruments incorporated components of previous questionnaires. The initial questionnaires (version 1.0) were distributed for evaluation in February 1995. Testing and refinement of these questionnaires led to the release of version 1.3 in 1996.
In 1995, the Academy, together with the specialty societies, conducted a pilot project to determine the feasibility of using the questionnaires to collect data from a variety of orthopaedic practices. The members of sixty orthopaedic practices agreed to participate in this project. These practices were selected to include a range of sizes, organizational structures, locations, and patient populations. Data collection was initiated on September 15, 1995, and concluded on December 15, 1995. Fifty-five of the practices completed the study. During the period of the study, 32,419 patient records were transmitted and 26,156 (80.7 percent) of them were entered into the database. Patients, orthopaedic surgeons, and practice managers were interviewed and were asked to fill out questionnaires concerning the project.
These evaluations and analyses of the records entered into the database showed that the collection of data from practices and the transmission of data to a central database were feasible. However, this study also demonstrated that the questionnaires, the methods of data collection, and the management of the database needed improvement. In general, patients had positive impressions of participation in the collection of outcomes data, especially when physicians or the office personnel explained the purpose of the data collection and physicians showed interest in the information collected. Patients had less favorable reactions when they were not provided with information concerning the purpose of the data collection and the physicians did not show interest in the information that the patients recorded. Slightly more than 60 percent (thirty-four) of the fifty-five practice managers indicated that the data collection made their job more difficult. About one-quarter (fourteen) of the fifty-five practice managers found that the project improved their interaction with the patients. Physicians and practice managers agreed that data collection imposed a burden on the practice and that the questionnaires and the mechanisms of data entry and transmission needed modification to decrease the time and resources necessary to collect and send data. Although the practices varied greatly in size and organization, it was clear that, in all of the practices, outcomes data could be collected most efficiently by integrating data collection into the established processes of patient evaluation and management.
As a result of the pilot data-collection project, the Academy and the specialty societies agreed to proceed with the development of a national database named MODEMS (Musculoskeletal Outcomes Data Evaluation and Management System)*. They also agreed that its purpose should be to improve the quality of care of patients who have musculoskeletal disorders by developing an information system that would make it possible to analyze the outcomes of musculoskeletal care and to provide participating physicians with valid and reliable methods of evaluating functional outcomes and the satisfaction of patients. Critical features of MODEMS include ownership and control of the database by physicians and the provision that information submitted by individual physicians or groups of physicians is available only to those individuals or groups.
In 1997, the Academy's Committee on Outcomes Studies completed the evaluation and revision of 2.0 versions of the four questionnaires. These versions of the pediatrics, spine, upper-extremity (or Disability of the Arm, Shoulder and Hand [DASH]), and lower-limb questionnaires have been incorporated into MODEMS. About fifteen minutes is required to complete each questionnaire. MODEMS also offers MODEMS HK (an instrument developed in collaboration with the American Association of Hip and Knee Surgeons to provide additional information concerning the outcome of operative management and the measurement of hip and knee function). Other researchers, specialty societies, and health-care organizations can develop supplemental modules based on the MODEMS questionnaires that will enable them to collect additional data of specific interest. This will result in high-quality studies that can provide valid and comparable data. MODEMS also provides data analysis and supports its use, and it provides data analysis for additional questionnaires that measure patient satisfaction, patient employment, and the physician's assessment of patients. In the near future, MODEMS will offer a short, generic musculoskeletal instrument, the Short Musculoskeletal Functional Assessment (SMFA), that covers spine and upper and lower-extremity conditions in adults in a single questionnaire.
In May 1997, the Musculoskeletal Education and Research Institute, an independent nonprofit corporation, was formed to promote musculoskeletal outcomes research and to assume responsibility for the MODEMS program. In addition, the Institute will provide specialized outcomes data-evaluation services to individuals and institutions and will disseminate research results. The Institute Board of Directors consists of physician-providers of musculoskeletal care, including neurosurgeons, rheumatologists, and orthopaedic surgeons; a consumer-representative member of the Arthritis Foundation; a representative of companies that produce products for the management of patients who have musculoskeletal disorders; and an expert in health-services research.
Since 1993, the efforts of the Academy and the Council of Musculoskeletal Specialty Societies to develop outcomes questionnaires and to establish a database have achieved notable successes. These include development of validated and reliable questionnaires and methods for transmission, analysis, and evaluation of outcomes data in a central database. MODEMS is operational, and the system ensures patient-physician confidentiality. The Musculoskeletal Education and Research Institute has assumed responsibility for MODEMS and for promoting outcomes research. The Institute is providing advice to and support for individual physicians and physician groups that want to evaluate their outcomes. In March 1998, the Institute released an inexpensive software program called Passport, which can be used with most office or personal computers to collect and transmit data obtained with MODEMS questionnaires. This program makes it possible for physicians to participate in MODEMS without purchasing additional computers or software. Currently, there are 249 physician-subscribers to MODEMS, and there have been more than seventy published research papers by authors who used questionnaires available through MODEMS. In addition, there are more than 100 research projects in which MODEMS questionnaires are currently being used.
Despite these successes, there are a number of problems facing the Institute. In particular, there is limited contribution to the database because the collection and analysis of outcomes data impose costs and burdens on patients and medical practices. In addition, many physicians are not convinced that regular, systematic evaluation of the outcomes of the care that they provide will improve their ability to manage patients. Currently, the Institute is developing methods that will make it easier and less expensive for physicians and patients to use the questionnaires. Moreover, the Institute is evaluating several projects that will investigate ways to use outcomes data to improve patient care and to devise better methods of reporting data to individual physicians that will help them to improve management of their patients. Success in these efforts will be critical for realizing the goal of the Academy and the musculoskeletal specialty societies to improve care of patients who have musculoskeletal injuries and diseases through use of outcomes evaluation.
Address for Dr. Buckwalter: Department of Orthopaedic Surgery, University of Iowa Hospitals and Clinics, 200 Hawkins Drive, Iowa City, Iowa 52242.
Outcomes data have the potential to be a powerful tool in clinical practice. Over the past ten years, outcomes research has demonstrated that important information about community-based clinical practice can be obtained8 and, furthermore, that this information can result in improvement in the quality and appropriateness of clinical practice.
Outcomes data can be collected at several levels: the individual physician or practice level, the state or regional level, and the national level. Each level has value, and the integration of data at all three levels presents the ideal situation. The methods and benefits of national data collection have been discussed, but it is important to point out that national organizations have an important role to play beyond that of data collection. They have the unique resources and capacity to develop validated outcomes questionnaires and other data-collection instruments and to advocate that these instruments become the standards for all investigators to use. The questionnaires from the Academy, the Council of Musculoskeletal Specialty Societies, and the Council of Spine Societies are examples of these efforts.
The need to use standardized and validated instruments to report the results of clinical research and assessment cannot be overstated. Only through the use of common instruments can reasonable comparisons of treatment methods, new technologies, and physician-based outcomes be made. Ideally, outcomes data systems should be in place at the practice, state, and national levels. Fortunately, methods to implement data collection are currently in place at all three levels. What is not as clear is whether the organizational infrastructures to facilitate these processes exist or will be developed. This discussion focuses on the methods and value of data collection and analysis at the state or regional level.
The Maine Medical Assessment Foundation (MMAF) is a nonprofit, physician-directed foundation for the improvement of patient care. Its activities are largely conducted through a series of physician specialty study groups. The work of these study groups spans a broad range of interrelated activities, including epidemiological analysis of medical-care utilization (small-area variations), externally funded clinical outcomes research, guideline development, implementation projects, and practice-based outcomes assessment. In many respects, the first four of these activities have provided the stimulus for the last.
Maine physicians have been motivated to participate in outcomes research and assessment as a result of the overall activities of the Maine Medical Assessment Foundation. The earliest work of the Foundation involved small-area analysis of health-care utilization in the state6. Once orthopaedic surgeons and others understood the importance and implications of variation in practice patterns in their own communities, they became willing participants in outcomes research projects developed through the Foundation. The projects involving musculoskeletal disease have included the Maine Lumbar Spine Study and the Maine Carpal Tunnel Study, which is now in its third phase of research.
Recently, an analysis of a medical-claims database that was conducted as part of the adaptation and implementation of the Agency for Health Care Policy and Research Low Back Pain Guideline2 provided striking evidence of how markedly different actual clinical practice is from the most basic recommendations of this guideline.
In 1995, the Maine Medical Assessment Foundation had a unique confluence of two sources of available data. The Maine Lumbar Spine Study had produced patient-oriented outcomes data from a group of enrolled patients who had been managed with lumbar disc excision1,6. The outcome of this study indicated that patients who had had an operation had been more severely symptomatic at the time of the initial evaluation and showed greater improvement at four years than did patients who had been managed nonoperatively1,7. At the same time, ongoing epidemiological analysis demonstrated prolonged and significant (p < 0.001) differences in population-based rates of lumbar discectomy among three distinct geographic areas in which the patients had had the operation (Fig. 1).
A new analysis was performed in which the patients were allocated to their specific so-called spine-service area. The analysis revealed that there was an inverse relationship between the outcome of the procedure and the area-specific rate of the procedure. In other words, patients in areas with lower operative rates had better outcomes than did patients in areas with higher rates (Fig. 2). Although patients in all three geographic areas had evidence of symptomatic herniation of a lumbar disc, those in areas with a high operative rate were shown to have less severe symptoms and signs of sciatica, a longer duration of symptoms, and less severe findings on imaging studies.
The response of Maine physicians to this report was revealing. Had there not been a tradition of data analysis and evaluation and participation in the outcomes study, it is likely that doctors from the high-rate areas would have reacted negatively to this information. They might have denied the validity of the small-area analysis and disputed the findings from the patient-reported outcomes study. However, none of these concerns was raised. Rather, the response was to request assistance in resolving the issues that were apparent in the findings. Surgeons from high-rate areas were honestly concerned about how to better understand and resolve the challenges raised by this analysis. As a result, almost all of the involved physicians from low, medium, and high-rate areas agreed to participate in the Outcomes Data Management Project that was developed by the Foundation.
The spine questionnaires developed by the Academy, the Council of Musculoskeletal Specialty Societies, and the Council of Spine Societies have been used for data collection. With use of methodology similar to that developed for the MODEMS program, patients are enrolled by the practices, completed questionnaires are computerized by the office staff, and the data are forwarded to the Foundation, where they are analyzed. Although the information is kept confidential, unlabeled reports are fed back to participants through the study group process and each physician receives his or her own confidential report. Currently, orthopaedists, neurosurgeons, osteopathic physicians, and chiropractors are collecting these data with use of the same questionnaire. There has even been a willingness of the specialty (spine service, osteopathic manipulative treatment, and chiropractic) study groups to share reports across the specialties.
This project is approaching the end of its first year. Consequently, definitive outcomes results are not yet available, but we are beginning to see early evidence of influence on practice. For example, rates of operations on the spine in Maine substantially increased from 1994 to 1996, just before the Outcomes Data Management Project began. Maine, like almost all areas, has had a substantial increase in the number of spine surgeons practicing in the state. The increase in operative rates is strongly associated with the increasing size of the workforce. In mid-1997, detailed data about the increasing operative rates were provided to the relevant study groups, and local hospital meetings were conducted when necessary. A recent analysis indicated a marked decrease in the rates of lumbar arthrodesis and lumbar discectomy in the second half of the year (Fig. 3). Additionally, practices are beginning to share and review physician-specific outcomes data internally.
This analysis illustrates the value of state-level data collection. The ability to integrate two different databases at the state level has produced new and valuable information that was not otherwise available.
Additionally, the presence of a state-based foundation to assist practicing physicians in addressing and beginning to resolve the challenges in a confidential, educational, nonregulatory model is an important component in these initiatives.
Although the example just described provides strong evidence of the value of the collection and analysis of data at the state level, there are several other reasons for these activities. Most orthopaedists practice in communities in which they do not have easy access to individuals who have research and analytical expertise. Such expertise may be important to assist practitioners in understanding the accuracy and importance of the information that they have collected. A less evident benefit of having an organization such as the Maine Medical Assessment Foundation participate in these efforts is that the data are analyzed and validated by a credible independent organization. This can eliminate any concern that there may be physician or practice bias in the analyses.
Data collection at the state level also offers the opportunity to share information across physician practices and different medical specialties, a step that the Maine Medical Assessment Foundation has been able to take. Finally, states are quite different from one another in many aspects of governmental, political, and economic conditions and health-care delivery systems. Efforts to understand how these factors interrelate with many aspects of medical care, including outcomes assessment, can only be undertaken at the state level. We believe that, as a result of the efforts of the Maine Medical Assessment Foundation, physicians have begun to understand and respond to the issues of variations in medical practice, which in turn has led to their participation in outcomes research, outcomes assessment, and other quality-improvement projects. Finally, practitioners have begun to integrate this kind of information in their daily practices—the ultimate goal8.
The collection of outcomes data at the practice (group), state, and national levels is technically feasible and desirable. In most areas, organizational and administrative barriers exist at each of these levels. Although not overwhelming, these barriers must be removed if such efforts are to be broadly successful. It would be ideal if all three levels of data collection could be integrated to provide a seamless flow of information from physician to state and national levels in both directions. For example, the Maine Medical Assessment Foundation has entered into a contract with the Musculoskeletal Education and Research Institute of the American Academy of Orthopaedic Surgeons. Eligible physicians in Maine are now able to collect data at the practice level and transmit the data to the Maine Medical Assessment Foundation; from there the data are sent to MODEMS. Thus, it is possible to produce an affordable and efficient system to accomplish these important goals.
Address for Dr. Keller: Maine Medical Assessment Foundation, P.O. Box 249, Manchester, Maine 04351.E-mail address: rbk@mmaf.org.
There are at least two goals of the MODEMS project. The first is to develop a national database on the outcomes of musculoskeletal procedures, and the second is to provide a method by which physicians can assess the results of musculoskeletal care in an outpatient setting and compare those outcomes with a national norm. Most health-care organizations have not systematically collected clinical data. This activity is even less common in physicians' offices. The stumbling block to performing outcomes studies has been the effort and cost required to enter the necessary data into a computer. In addition, physicians have been reluctant to purchase computer hardware for the purpose of doing clinical studies. Unlike hospitals, which have purchased computer systems that can be used for multiple functions, computer usage in physicians' offices has generally been limited to billing systems.
Nevertheless, many financial databases are rich resources of information that can be analyzed to yield insight into clinical performance. This assessment can be done in one of two ways. The billing software can be altered to generate the appropriate standard reports. This usually requires an additional fee to the software user. Alternatively, the vendor can write a computer program that downloads the data into one of the standard spreadsheets, such as Lotus 1-2-3 or Microsoft Excel, or into database-management packages, such as Dbase or Access. With an inexpensive personal computer and limited knowledge of these software packages, physicians can evaluate the information routinely with little cost or additional effort. The advantage of the second alternative is the capability of a much richer analysis.
In most office settings, the office personnel send a bill by completing a Health Care Financing Administration (HCFA) Form 1500, thereby entering the data into the computer. This form includes fields for socioeconomic data such as the name, address, and social security number of the patient and of the insured individual, the relationship of the patient to the insured individual, the insurance carrier, and the charges. In addition, there are fields into which clinical data such as diagnosis and procedure codes, date and place of service, and referring and treating physician can be entered. Although we cannot assess actual outcomes from these data, there is ample information to give physicians a good overview of a practice.
The three kinds of information that are available from this resource are utilization of health-care services, practice patterns, and production. Examples of useful utilization information that is available are the number of visits by physician, in total or for a particular diagnosis; the number of radiographs made by a physician for a particular diagnosis; the total charges by diagnosis; the number of procedures by site of service; and the duration of inpatient stay by diagnosis or procedure. Information on practice patterns might include the number of patients by zip code, referring physician, employer, or insurance carrier. Examples of production information include the number and type of procedures by physician, office, or site; the charges by physician; and the charges by period (week, month, or year). This information can be matched with diagnoses and procedures. By including a database that matches current procedural terminology (CPT) codes and the corresponding relative value unit (RVU), physicians can determine production by relative value unit instead of by charges or collections. Because charges and collections rarely match, the relative value unit is a more accurate assessment of work by each physician, site, or office.
MODEMS will have three distinct parts. The first and largest (and most difficult to implement) is the outcomes database. The second is the patient-satisfaction survey that will be available separately. The third is the HCFA Form-1500 information discussed earlier. There is a distinct advantage to performing a practice assessment with MODEMS. MODEMS uses a unique patient identifier that is a combination of part of the patient's social security number, the patient's birth date, and the local office-identification number. This method of identification protects patient confidentiality, as the only individuals who can identify the patient are the personnel in the physician's office. However, it also means that by using the MODEMS project in its entirety, a practice will be able to match patients in all three databases and compare outcomes, utilization of health-care services, and patient satisfaction. I think that this will be the only database in an outpatient setting that has this capacity.
In summary, the orthopaedic community has been the leader in developing validated tools to perform outcomes research and methods of collecting and analyzing data generated by the use of these tools. In addition, individuals within the orthopaedic specialty societies have been effective in educating orthopaedic surgeons concerning the use of these data to improve clinical practice. Statewide orthopaedic networks such as the one established in Maine have been demonstrated to be effective frameworks for collecting and using data. The value of outcomes data to groups and individual surgeons seems clear; however, the requirements of cost and personal involvement are considerable. Efforts to increase the efficiency and to decrease the costs of data collection and analysis must continue. For now, the orthopaedic community seems best advised to further develop the large orthopaedic group-practice, state, and subspecialty networks to collect meaningful data. By working together to identify the clinical issues of relevance and identifying the sources of expertise for study coordination, we will best serve our patients. In many cases, this will be in association with not-for-profit foundations and university-based departments and colleges.
Address for Dr. Haralson: Maryville Orthopaedic Clinic, P.O. Box 5479, Maryville, Tennessee 37802. E-mail address: rharal5653@aol.com.
*For more information on MODEMS, contact Beth Imrem, Data Harbor, 221 West Walton, Chicago, Illinois 60610. Telephone: 800-228-0018. Fax: 312-944-1507. E-mail: bvi@dataharbor.com. Web site: www.modems.com.
Atlas, S. J.; Deyo, R. A.; Keller, R. B.; Chapin, A. M.; Patrick, D. L.; Long, J. M.; and Singer, D. E.: The Maine Lumbar Spine Study, II. 1-year outcomes of surgical and nonsurgical treatment of sciatica. Spine,21: 1777-1786, 1996.211777
1996
[PubMed]
Bigos, S. J. [Chairman]: Acute Low Back Problems in Adults, Clinical Practice Guideline Number 14. AHCPR Publication 95-0643. Rockville, Maryland, United States Department of Health and Human Services, Public Health Services, Agency for Health Care Policy and Research, 1994.
Codman, E. A.: The Shoulder. Rupture of the Supraspinatus Tendon and Other Lesions in or About the Subacromial Bursa, pp. 1—29. Malabar, Florida, Robert E. Kreiger, 1984.
Gartland, J. J.: Orthopaedic clinical research. Deficiencies in experimental design and determinations of outcome. J. Bone and Joint Surg.,70-A: 1357-1364, Oct. 1988.70-A1357
1988
Gross, M.: A critique of the methodologies used in clinical studies of hip-joint arthroplasty published in the English-language orthopaedic literature. J. Bone and Joint Surg.,70-A: 1364-1371, Oct. 1988.70-A1364
1988
Keller, R. B.; Soule, D. N.; Wennberg, J. E.; and Hanley, D. F.: Dealing with geographic variations in the use of hospitals. The experience of the Maine Medical Assessment Foundation Orthopaedic Study Group. J. Bone and Joint Surg.,72-A: 1286-1293, Oct. 1990.72-A1286
1990
Keller, R. B.; Atlas, S. J.; Singer, D. E.; Chapin, A. M.; Mooney, N. A.; Patrick, D. L.; and Deyo, R. A.: The Maine Lumbar Spine Study, Part I. Background and concepts. Spine,21: 1769-1776, 1996.211769
1996
[PubMed]
Keller, R. B.; Wennberg, D. E.; and Soule, D. N.: Changing physician behavior: the Maine Medical Assessment Foundation. Qual. Manag. Health Care,5: 1-11, 1997.51
1997
Sledge, C. B.: Crisis, challenge, and credibility. J. Bone and Joint Surg.,67-A: 658-662, April 1985.67-A658
1985
Wennberg, J. E.; Bunker, J. P.; and Barnes, B.: The need for assessing the outcome of common medical practice. Ann. Rev. Pub. Health,1: 277-295, 1980.1277
1980
Wennberg, J., and Gittelsohn, A.: Variations in medical care among small areas. Scient. Am.,246: 120-134, 1982.246120
1982