Extract
The American Board of Orthopaedic Surgery (ABOS) is one of the twenty-four
separate boards that make up the American Board of Medical Specialties. Of the
twenty-four boards, fourteen require an oral examination. At present, the ABOS
is the only board with a computerized data collection system that allows for
the analysis of the collected information in the database.The ABOS exists to serve the interests of the public and the medical
profession by establishing educational standards for orthopaedic residents and
by evaluating the initial and continuing qualifications and competence of
orthopaedic surgeons. Upon completion of an approved residency program,
applicants for board certification must pass a comprehensive, proctored
written examination (the Part-I examination). Applicants must then practice
orthopaedics for twenty-two months, twelve of which must be in one location.
Many elect to begin the practice requirement following a fellowship. The
applicants must then satisfactorily complete a thorough credentialing process
and pass an oral examination (the Part-II examination) that is based on all of
the candidate's operative cases in six consecutive months beginning one year
before the oral examination.
The American Board of Orthopaedic Surgery (ABOS) is one of the twenty-four
separate boards that make up the American Board of Medical Specialties. Of the
twenty-four boards, fourteen require an oral examination. At present, the ABOS
is the only board with a computerized data collection system that allows for
the analysis of the collected information in the database.
The ABOS exists to serve the interests of the public and the medical
profession by establishing educational standards for orthopaedic residents and
by evaluating the initial and continuing qualifications and competence of
orthopaedic surgeons. Upon completion of an approved residency program,
applicants for board certification must pass a comprehensive, proctored
written examination (the Part-I examination). Applicants must then practice
orthopaedics for twenty-two months, twelve of which must be in one location.
Many elect to begin the practice requirement following a fellowship. The
applicants must then satisfactorily complete a thorough credentialing process
and pass an oral examination (the Part-II examination) that is based on all of
the candidate's operative cases in six consecutive months beginning one year
before the oral examination.
Before 1998, candidates submitted individual case lists on paper. Beginning
in July 1998, candidate case lists were collected as part of a nationwide
orthopaedic database as a way of organizing the case lists for review and case
selection for oral examination. The computerized lists can be analyzed for the
entire candidate group or subgroups. This yields a comprehensive dataset of
approximately 700 candidates per year.
Candidates for certification enter all operative procedures performed in
six consecutive months in the year prior to examination. The candidate's case
lists are verified from each hospital or surgical center during the six-month
period. The case list from each hospital or surgical center must be notarized
by the director of medical records. The ABOS has contracted with an outside
vendor (Data Harbor Solutions, Hinsdale, Illinois) to develop and manage the
databases. During the first year of data collection (1998), floppy disks were
used. Subsequently (beginning in 1999), data were submitted by means of a
secure password-protected web-based interface. Once completed, the case lists
are reviewed by practicing ABOS-certified orthopaedic surgeons working with
the ABOS. From the entire case list submitted by the candidate, twelve cases
are selected for review at the Part-II oral certification examination. The
candidates are required to submit medical records, radiographic studies, video
or photographic prints from cases of patients managed with arthroscopy, and
data with regard to outcomes and complications. Candidates bring materials and
accompanying information for ten of the selected cases to the examination. The
full case list, as well as aggregate practice summary data, is available for
the oral examiners.
The database used by the Part-II candidates provides a relatively complete
and accurate assessment of practice over a six-month period. There are strong
motivations for the candidate to provide an accurate list of cases, and
safeguards are built into the system. Still, there is a potential for bias in
the reported data if there were a conscious alteration of practice patterns by
candidates during the surgical list collection period. The online system
limits the possibility of invalid or incomplete data through the use of
drop-down menus, look-up tables, and enforcement of the requirement for
complete records prior to final submission of the case list.
The case lists include the diagnosis (International Classification of
Diseases, Ninth Revision [ICD-9] codes), age, sex, procedures performed
(Current Procedural Terminology [CPT] codes), and date of surgery for
each patient. Information with regard to death, complications, and outcome for
each patient is mandatory. Reporting the occurrence of anesthetic
complications; specific surgical and/or technical complications; specific
medical and/or systemic complications, including death; and surgeon-reported
outcomes of pain, deformity, function, and patient satisfaction is mandatory.
No uniform method of reporting these data is in use, although the ABOS has
pilot-tested an outcome instrument to better standardize outcome data. Today,
the candidate and patient burden have been deemed too high for this
purpose.
The number of cases and procedures from 1999 through 2003 are listed in
Table I-A. The table provides
pooled information, including the number of applicants, surgical cases on
individual patients, and total number of procedures (a given patient may have
had more than one procedure performed during a single surgical case).
Table I-B shows the average
number of applicants according to their self-reported subspecialty. The
declared subspecialty helps the Board to choose oral examiners. Declaration
does not indicate whether the candidate has completed a fellowship. Rather,
the choice of a subspecialty allows the candidate and the examiners to be
matched in areas of practice and expertise.
The number of cases by deciles is listed in
Table II. The median range was
107 to 120 cases for five years. This table is important as it shows not only
the median surgical output but also the minimum and maximum number of cases
for the six-month case list for 1999 to 2003.
The patient gender mix has been steady at approximately 54% male and 46%
female.
Cases also may be reviewed by procedure codes (CPT) and/or diagnoses
(ICD-9) (see Appendix). The top twenty-five individual procedures performed
are listed in Table III. Three
of the top four procedures involve arthroscopy codes: two knee and one
shoulder procedure. Total knee and total hip arthroplasty ranked seventh and
twelfth, respectively, in 2003.
Trends in the number of procedures from year to year are a reflection of
changes in practice for Part-II candidates
(Table III). Partial excision
of the medial or lateral meniscus of the knee (CPT code 29881) remains the
most common procedure, followed by carpal tunnel surgery (CPT code 64721).
Shoulder arthroscopy and/or acromial decompression (CPT code 29826) moved up
seven places over the five years. Absolute numbers of procedures, their rank
order, and changes over time also are shown in
Table III. The total number of
procedures coded 29881 (meniscectomy) is consistently more than twice that of
procedure 64721 (carpal tunnel release), which, until 2003, was the second
most common procedure. Five of the top eleven procedures required the use of
arthroscopy skills. When the absolute numbers are summed for these procedures,
56% of the top eleven procedures required arthroscopy.
Table IV lists the top
twenty-five ICD-9 diagnosis codes. The practicing surgeon is aware that a
single CPT code can be associated with more than one ICD-9 code and,
similarly, an ICD-9 code can be associated with different surgical procedures
or CPT codes. The ICD-9 codes for knee menisci were two of the top three codes
reported. Similar to the data in Table
III, the top five codes are consistent over the five years
reported in the present study.
The Board can also monitor the average number of procedures and the
percentile ranking for each of the most frequently performed procedures. For
example, the mean number of cases per applicant for CPT code 29881 (knee
arthroscopy/partial meniscectomy) was 11.45 cases in 2003. The median number
(a figure unaffected by extremes) in the most recent three years reported was
consistently found to be eight cases (see Appendix). For code 29881, the
maximum number of cases for an applicant in that six-month period for 2001 and
2002 was sixty-eight and fifty-nine, respectively, or about ten per month.
However, for 2003, one applicant reported nearly double that number.
The top four spine CPT codes listed were among the top thirty-five
procedures performed (see Appendix). However, 75% of the applicants
participating in Part-II Board certification did not perform any of these four
spine procedures. In 2003, seventy applicants listed their subspecialty as
spine; there were 1501 lumbar spine fusions performed (935 CPT 22612
posterior, posterior lateral; 347 CPT 22630, posterior interbody technique;
and 219 CPT 22558, anterior interbody technique). While average case volume
numbers are not needed to evaluate a candidate's specific practice pattern,
sometimes a broad view provides valuable information. The average number of
procedures based on the number of declared spine specialists yields 13.4
posterior lumbar fusions (CPT 22612), five posterior interbody fusions (CPT
22630), and three anterior interbody fusion procedures (CPT 22558) during the
six-month period prior to their application. Such data can provide potentially
interesting information on practice patterns to any surgeon who is just
starting practice. Obviously, practice patterns often change as a surgeon's
practice matures. These collated data also can be used to provide valuable
information about the use of specific orthopaedic procedures. For example,
Figure 1 shows that CPT 29888
(anterior cruciate ligament surgery) is being performed on predominantly young
patients, with a peak volume among those of high-school age and then a decline
in young adults. Meniscectomy extends over a larger age range
(Fig. 2, CPT 29881), while the
rate of chondral surgery procedures (Fig.
3, CPT 29887) gradually increases to the age of fifty. Similar
comparisons can be made for any coded procedure.
These data provide orthopaedic surgery residents and educators accurate
information about the types and volumes of procedures that young orthopaedic
surgeons are likely to perform in the first two years of their practice. They
can expect to perform around 120 cases over a six-month period
(Table II), on mostly
middle-aged patients (Table
III), and many involve the use of arthroscopic procedures (five of
the top eleven CPT codes, Table
III).
The Board can also compare the average volume of procedures performed by
the candidate pool with the number of procedures performed by any individual
candidate. For example, the applicants for 2001, 2002, and 2003 averaged about
eleven knee arthroscopies during their six-month case-collection period. The
50th percentile figure (median) was consistently eight, meaning that 50% of
the applicants performed fewer than eight knee arthroscopies and 50% performed
more than eight knee arthroscopies. An extreme number posted by a candidate
might be indicative of a new surgeon stepping into a specialty practice, or it
might suggest a difference in the individual's indications for that operation
compared with the larger orthopaedic community. The Board serves the public
interest and must review extreme counts on either end of the spectrum.
Candidate case lists are expected to be accurate for the six-month period
of collection. The Board is unable to extrapolate beyond this time.
Maintenance of Certification, the new process of recertification, also
involves a case list. This will provide another opportunity to view current
practice patterns.
Lifelong learning is important in our profession. The ABOS recognizes
candidate anxiety and the time necessary to prepare for each examination. The
ABOS also realizes the value inherent in the process of preparing for these
examinations. It is the goal of the Board that orthopaedic patients benefit
from this rigorous examination and peer review.
The ABOS has worked for many years to understand and to improve the Part-II
oral examination, and the orthopaedic oral examination process has actually
been a model for several other surgical specialty board examinations. The test
has been determined to be as psychometrically valid as the initial written
examination by Measurement Resources (Chicago, Illinois), a psychometric
educational consultation firm used by most of the American Board of Medical
Specialties Surgical Specialty Boards who employ an oral examination as part
of their certification process.
These data are of obvious importance in the testing of candidates for
certification by the ABOS. The aggregate data are also of use in determining
what conditions recent graduates of residency programs are seeing and what
procedures they are performing in practice. They also may be used to inform
the Board and the profession in general of changes in practice patterns over
time.
Additional queries to the database may provide details about regional
differences in practice, what fellowship-trained surgeons are doing compared
with non-fellowship-trained candidates, the breakdown of procedures for the
general orthopaedist compared with the subspecialist, and complications
associated with treatment of various diagnoses. This is clearly demonstrated
for spine surgery, wherein 75% of the candidates in the study reported
performing none of the top four spine surgeries.
This information provides an interesting database for health-care research
and policy priorities for orthopaedic surgery. These data provide advantages
over other secondary databases because they are more accurate with regard to
surgeon location and procedures performed and are verified by the candidates
taking the examination. The disadvantages of the database include the
relatively young age of the surgeons and the limited years of practice after
training. Thus, these data are predominantly reflective of surgeons early in
practice as opposed to those whose practice has matured. Clearly, these data
can be a valuable addition to other available databases such as the National
Hospital Discharge Survey, the National Ambulatory Medical Care Survey, the
National Health Interview Survey, and the Longitudinal Studies on Aging (all
of which are maintained by the National Center for Health Statistics).
It should also be pointed out that candidates may not practice in the same
manner during the time that the cases are collected for Part-II certification.
For example, controversial treatment methods or very difficult cases might be
avoided by the candidates. The data are still accurate but may not be totally
representative of the candidates' practices at times when data are not being
collected.
Part of the charge of the research committee of the Board is to explore
ways to better utilize this database for quality improvement and safety in
orthopaedic practice and to provide information to examination candidates and
diplomates that improves practice. New queries of the dataset should benefit
the Board, our examination candidates, orthopaedic surgeons in practice, and
the overall practice of medicine, to the ultimate benefit of our patients.
In conclusion, the database of a useful research tool for orthopaedic
surgery-related health research. It is a valuable source of information on the
practices of the examinees and may well be extremely useful in the design of
residency education, the initial written examination for ABOS certification,
and, ultimately, in the evaluation of practice performance.
Tables presenting descriptive statistics from the top ten CPT codes and the
top four spine CPT codes, and the top procedure codes for spine subspecialists
are available with the electronic versions of this article, on our web site at
(go to
the article citation and click on "Supplementary Material") and on
our quarterly CD-ROM (call our subscription department, at 781-449-9780, to
order the CD-ROM).