Abstract
Background: A recent survey of medical and surgical residents in the
United States suggested that our current training of physicians may be
inadequate to meet the increasing demand for diagnosis and treatment of
musculoskeletal disorders. In response, we developed an integrated,
multidisciplinary course to teach knowledge and skills related to
musculoskeletal disease to second-year medical students. A three-year
prospective outcomes study was conducted to evaluate the new course.
Methods: The primary outcomes that were studied during the first
year of the new course were the gains in knowledge, changes in levels of
confidence, and long-term retention of skills. Secondary outcomes consisted of
student and faculty satisfaction. Ten-item pre-tests and post-tests covering
core course concepts were administered to students. A matched-pairs t test was
used to evaluate the difference between pre-test and post-test scores.
Students were also asked to rate, on a 10-point scale, how much confidence
they had in their ability to perform the musculoskeletal physical examination
before and after the institution of the new curriculum. A general linear model
analysis with post hoc pairwise comparisons (F test) was used to evaluate the
changes in the confidence levels of the students. Also, a knee examination
station was organized to compare students' scores before and after revision of
the course. At the conclusion of the course, students rated each aspect of it
on a scale of 1 to 5. Instructors were asked to rate the effectiveness of all
elements of the course on the same scale.
Results: On the basis of student satisfaction and confidence and
faculty satisfaction, the most effective changes in the curriculum were the
introduction of a physical examination workshop and simulated clinical
situations. Students' knowledge increased significantly (p < 0.001), and
their level of confidence increased significantly in thirteen specifically
targeted areas (p < 0.0001). On the end-of-the-year examination assessing
retention of physical examination skills, the scores for the skills emphasized
in the revised course increased significantly whereas the scores for a skill
not emphasized in the course remained the same. Revisions made in the second
and third years after implementation of the course expanded the more
successful elements and further improved student ratings.
Conclusions: Integration of the three clinical disciplines related
to musculoskeletal disease—orthopaedics, rheumatology, and physical
medicine and rehabilitation—resulted in a highly effective introductory
course for second-year medical students. The heuristic strategy of introducing
core content through lectures and workshops followed by small-group teaching
sessions for practice with the new knowledge effectively increased students'
knowledge, confidence, and satisfaction.
Musculoskeletal problems rank first as a reason to seek medical care, and
they consume a large portion of medical
resources1.
According to Yelin and Callahan, in 1992 2.5% of the gross national product in
the United States ($149 billion) and 3.5% of the gross national product in
Canada ($25.6 billion) were spent on musculoskeletal
ailments2. According
to the Centers for Disease Control and Prevention, by 2020 the number of new
patients with arthritis in the United States will have increased more than the
number of patients with any other
disease3.
A recent survey of second-year medical and surgical residents in the United
States suggested that their current education and training may be inadequate
to meet this increased
need4. Seventy-eight
percent of the respondents thought that they were not prepared to conduct a
musculoskeletal examination. One reason for this lack of confidence may be the
small amount of time spent on musculoskeletal education in the typical medical
school curriculum. A recent analysis of sixteen Canadian medical schools
revealed that, on the average, only 2.26% (range, 0.61% to 4.81%) of the
undergraduate curriculum was spent on musculoskeletal
education5. This is
in marked contrast to the actual practice of family physicians, who spend an
average of 13.7% to 27.8% of their clinical activity dealing with symptoms
directly related to the musculoskeletal
system5.
At our institution, medical students learn to apply basic science to
clinical medicine in a yearlong pathophysiology course during the second year.
The course is organized by organ systems, one of which is entitled
"Bones, Joints, and Connective Tissues." However, the course work
related to the musculoskeletal system was fragmented, and both student and
faculty reported that the students had a suboptimal level of basic knowledge
of musculoskeletal problems during third and fourth-year clerkships.
Therefore, we developed an integrated multidisciplinary second-year
musculoskeletal disease course and prospectively assessed the impact of this
curriculum change over three years.
Course Development
Before implementing the new course, we surveyed the student body after they
had completed the orthopaedic and rheumatology courses in 2000. We asked the
students to rate the overall course structure, organization, clarity of
objectives, and lecture quality. On the basis of these scores, we held
focus-group discussions with student representatives and course faculty. With
this feedback, we moved toward an integrated musculoskeletal course.
The next step was to align the various departments involved to commit their
individual course time to an integrated multidisciplinary musculoskeletal
disease course. This integrated approach empowered faculty organizers to
negotiate with other course directors to release curriculum time for the new
course. For example, we were able to increase the course by five hours by
negotiating with directors of the second-year family medicine rotation to
release students for the musculoskeletal physical examination workshop
session.
Course leaders from the Departments of Anatomy, Orthopaedics, Rheumatology,
and Physical Medicine and Rehabilitation met weekly to develop an integrated
musculoskeletal curriculum focused on the primary care setting. We were guided
by the principle that this course should emphasize diagnosis and
pathophysiology. Our basic teaching strategy was to use different formats to
present the core content, following a logical progression from lectures to
multidisciplinary case presentations and finally to hands-on learning in
small-group sessions. The goal of the lectures was to cover course core
material in a didactic fashion. The multidisciplinary case presentations were
designed to provide the same core material in another format, highlighting the
overlap of orthopaedics, rheumatology, and rehabilitation medicine. There were
two types of small-group sessions: one was designed to advance physical
examination skills (the physical examination workshop), and the other was
designed to allow students to practice their new knowledge and physical
examination skills in simulated clinical settings, which we called
"bellringer sessions." In addition, students were encouraged to
use web-based interactive cases for independent study.
The number of lectures was reduced. The course directors selected the
lecturers who had received high ratings from students the year before (2000)
to participate in the new course. Traditional small-group sessions for
case-based discussions had had mixed success in the past, primarily because of
variations in teaching style among instructors. Those sessions were replaced
with theater-style multidisciplinary case presentations to ensure uniformity
of curriculum content for all students. The faculty were mentored by the
course directors to develop their presentations in a uniform PowerPoint format
(Microsoft Office 2000; Microsoft, Redmond, Washington) so that they could be
placed on the course web site. The mentoring process provided the opportunity
for the course directors to have direct input in the lecture content, improve
the quality of illustrations, and ensure integration of the material
presented.
The topics covered in the lectures are presented in
Table I. The four
multidisciplinary presentations were designed to illustrate and to expand on
the content presented in the lectures. They included (1) the diagnosis and
natural history of rheumatoid arthritis, complicated by an acute knee injury
requiring diagnosis, and treatment of end-stage rheumatoid arthritis with a
total knee replacement (two hours); (2) hip fracture and the associated issues
of osteoporosis, osteoarthritis, and prosthetic hip replacement (two hours);
(3) lupus, to illustrate various problems in diagnosis and treatment (one
hour); and (4) musculoskeletal trauma due to a motor-vehicle accident, to
address the initial management of multiple fractures, open fractures, and
joint infection and compartment syndrome (one hour).
Small-group sessions, with six, seven, or eight students per instructor,
were used to teach the musculoskeletal physical examination and to identify
common regional pain problems. These small-group physical examination
workshops were conducted in a gymnasium for hands-on instruction, with use of
an outline of basic examination techniques for the upper and lower extremity
and identification of regional pain problems. Groups of students rotated
through stations, where they observed and then practiced examination of the
hand, wrist, elbow, shoulder, hip, knee, and foot. We specifically did not
include examination of the back because this region is taught in our
fourth-year curriculum. Each student spent a total of five hours in the
physical examination workshop.
At our institution, the second-year medical school class has 165 students.
Therefore, to keep the groups small, we invited community physicians from the
fields of orthopaedics, rheumatology, and rehabilitation medicine to
participate in these groups. The instructors donated their time to this
activity without compensation.
Before the start of the new course, we wanted to standardize the quality of
instruction in the small-group sessions. This had been identified as a weak
component in previous versions of the course and a source of concern during
focus-group discussions among students and instructors. In preparation for the
small-group sessions, instructors were convened beforehand to review the
workshop objectives and the material that needed to be covered in order to
decrease instructor variability. We also standardized the teaching by having
the instructors teach one region of their interest to rotating groups of
students.
The bellringer sessions were conducted after the physical examination
workshop, with groups of six, seven, or eight students who rotated through a
total of five stations, with two faculty members at each station, on each of
two afternoons. Each station was set up in duplicate to keep the groups small.
At each station, the students were presented with a problem and asked to
arrive at a diagnosis, develop a treatment plan, or demonstrate a physical
examination technique. After thirty minutes, a bell rang to signify that it
was time to move on to the next station. These sessions were designed to
utilize the knowledge taught in the lectures, the multidisciplinary case
presentations, and the physical examination workshop. Each student spent a
total of five hours participating in bellringer sessions. The stations and the
clinical problems are described in Table
II.
Four interactive web-based cases illustrating different disorders causing
inflammatory arthritis were also developed for independent study. The
interactive format was designed to provide immediate feedback to the student
as he or she progressed through the diagnostic workup. Each step provided
historical, clinical, and laboratory information about the patient followed by
multiple-choice questions, which led the student to the next step. Immediate
feedback on each option was provided through linked fields containing the
explanation of correct and incorrect responses.
Course Assessment
The primary outcomes studied during the first year of the new course were
the gains in knowledge, changes in levels of confidence, and long-term
retention of skills as assessed with the knee examination. Secondary outcomes
consisted of student and faculty satisfaction with the new course. To assess
the gains in knowledge, a ten-item test was administered to the students prior
to the start of the course and at its conclusion. Test items were drawn
randomly from a pool of questions covering core course concepts. A
matched-pairs t test was used to measure the difference between pre-test and
post-test scores.
Students were asked to rate how much confidence they had in their ability
to evaluate musculoskeletal problems at three points during the course: before
the physical examination workshop (baseline), after the workshop, and after
the bellringer sessions. All items except the pediatric hip examination and
interpretation of synovial fluid tests were presented before the small-group
sessions. During the physical examination workshop, the students specifically
did not receive instruction on the pediatric hip examination, but they were
asked to rate their confidence in performing it in the post-workshop
questionnaire. After the physical examination workshop and during the
bellringer sessions, we provided formal instruction on the pediatric hip
examination and again asked the students to rate their confidence after those
instructional sessions. This was done to validate our assessment of student
confidence as a measure of knowledge and to rule out the possibility of a
confounding effect from the small-group sessions, which could have biased our
results. As students engage in initial clinical experiences, a surge in
enthusiasm can easily translate into an increase in confidence that might be
generalized to all items queried. The lack of change for this negative
controlled observation would be an important validation of changes in levels
of confidence as an outcome measure for this educational intervention.
Students rated their levels of confidence on a 10-point scale, ranging from
none (1) to high (10). A general linear model analysis with post hoc pairwise
comparisons was used to evaluate the changes in the students' levels of
confidence. The F test was used to derive the ratio of two squares and to
estimate population
variance6. The F
test provides a test for significance of observed differences among the means
of two or more random samples.
Students' Retention of Skills
A twenty-two-station physical examination test is administered to all
students at the end of their second year. The retention of physical
examination skills and knowledge was prospectively assessed during this
session, twelve weeks after course completion in 2001 and sixteen weeks after
course completion in 2002. A knee examination station that tested material
taught in our new course was established. Students' performance at this
station was evaluated by a trained (blinded) evaluator using a standard
checklist rating form with sixteen skill and knowledge items and five
interpersonal skill items (e.g., "establishes rapport with the
patient"). The station was organized to test retained knowledge and
physical examination skills. The scores for the students' performance at the
knee station were collected prospectively one year prior to the implementation
of the new course (i.e., in 2000) and served as a control with which to assess
the overall effectiveness of this new curriculum. T tests were used to compare
the students' scores at the knee examination station in the year before and
the year after the revision of the course.
Secondary Outcomes: Student and Faculty Satisfaction
At the conclusion of the course, students rated each aspect of the course
on a scale of 1 (excellent) to 5 (poor). Instructors were asked to rate the
effectiveness of all of the elements of the course on the same scale.
The students' knowledge about musculoskeletal problems had increased
significantly at the conclusion of the course. On the pre-test, students
answered a mean (and standard deviation) of 2.65 ± 1.41 of ten items
correctly. On the post-test, they answered a mean of 6.64 ± 0.96 items
correctly. This difference was significant (t = 26.49, p < 0.001).
The students' levels of confidence in their overall ability to evaluate
musculoskeletal problems increased, after they participated in the physical
examination workshop, from 3.87 to 5.38 (p < 0.0001) (see Appendix). The
level of confidence significantly increased for twelve of the thirteen items.
As expected, the level of confidence for the pediatric hip examination, which
was not presented in the physical examination workshop (but was presented
subsequently in one of the bellringer sessions), was unchanged after the
physical examination workshop (4.44 before the workshop compared with 4.67
after it).
Following the bellringer sessions, the students' levels of confidence
increased further, from 5.38 after the physical examination workshop to 6.84
(p < 0.0001) (see Appendix). The level of confidence significantly
increased for all thirteen items (p < 0.0001). After the bellringer
sessions, the level of confidence for the pediatric hip examination increased
to 6.38 (p < 0.0001).
The scores for retention of skills, based on the number of items
successfully completed on the checklist, increased from 71% in the year prior
to the new course (2000) to 83% in 2001 (t = 7.56, p < 0.001) and to 74% in
2002 (t = 2.07, p < 0.04). The students' general interpersonal skills,
which were not targeted in the revised musculoskeletal course, remained the
same, with a mean of 4.76 (of 5) in the year before the course was revised and
a mean of 4.65 for the class that took the revised course (t = 1.68, p <
0.094). During a debriefing session, the students attributed their performance
on the knee examination to the new format of the musculoskeletal course.
The students' mean overall rating of the new course, after its first year,
improved to 2.27 (of 5) compared with a rating of 2.59 for the previous
year's, unrevised course (see Appendix). The musculoskeletal physical
examination workshop and the bellringer sessions were the most positively
rated elements of the course, with mean ratings of 1.91 and 1.48,
respectively. Mean ratings of the didactic lectures improved modestly, from
2.50 in 2000 to 2.31 in 2001.
The administrative and organizational aspects of the course received
average ratings, indicating the need for additional improvements. The
multispecialty case presentations were not well received by the students, who
reported that they were too long and complex and that they included material
not presented in the preceding didactic lecture. The use of a web site for
course administration (schedules, course component descriptions, and
small-group assignments) was not efficient for the students and caused
confusion and frustrations because of browser incompatibilities. Provision of
the course syllabus on the web site was soundly criticized by the students who
wanted to have a paper copy of the entire course syllabus in hand at the start
of the course. Despite the enthusiasm for web-based education, online cases
for independent study received low ratings because of substantial technical
problems that limited student access.
On the basis of these outcomes after the first year of the course, lectures
were refined, formats were modified, and the reliance on web-based materials
was reassessed. In the second year, the six hours that had been used for the
theater-type multidisciplinary cases were used for additional lectures on
osteoarthritis and vasculitis, and an additional bellringer session was added
to reinforce the learning of didactic material. The clinical anatomy review
was not as effective as we had hoped, and, in the second year, this material
was integrated into the lectures and the bellringer sessions. Course directors
reviewed all lectures in preparation for the second and third years. Some
lectures were extensively revised, and the Power-Point format was fully
implemented. For example, two faculty members worked together to integrate the
lectures on mechanical back pain and spondyloarthropathies to effectively
present the differences in symptoms, physical findings, and radiographic
changes in order to eliminate redundancy and to avoid presenting contradictory
statements to the students.
Repeated outcomes evaluations of the revised course as it had been given in
the second and third years revealed improvements in the overall rating of the
course as well as the ratings of the course organization, the lectures
overall, and the clarity of objectives (see Appendix). New components of the
integrated course—namely, the physical examination workshop and the
bellringer sessions—continued to be given high ratings and the rating of
the web page improved slightly in 2002, but in 2003 the rating was worse than
it was in 2001. The bellringer sessions remained the most highly rated
component of the course. Over the three years, the overall student evaluation
of the course improved the course ranking from eleven of the twelve
second-year courses to five of the twelve.
Instructors rated the effectiveness of the physical examination workshop
for teaching skills as 2.2 and that of the bellringer sessions as 1.8. They
were enthusiastic about the new formats and rated the effectiveness with
regard to building rapport with students as very high. Additional proof of
faculty acceptance of the changes was the 90% return rate for the following
year and an increase in the volunteer pool of potential instructors over the
three years.
Changes in curriculum content and format are often driven by faculty or
school-based factors, and the impact of such changes on students may not be
measured. This study demonstrates that the impact of major changes in the
musculoskeletal disease course for second-year medical students can be
measured in terms of the students' gains in knowledge, change in confidence,
and retention of physical examination skills. Prior to implementation of the
integrated course, faculty were assigned a lecture topic and developed the
content and presentation style in isolation, without input from the course
directors. There was reluctance and skepticism about the possibility of peer
review of individual faculty lectures. Our experience demonstrates that a
process of course integration can provide an effective opportunity for the
course directors to improve lecture and course quality. Changes in teaching
formats within courses have enormous consequences in terms of faculty time and
cost.
The most impressive result of this experiment was the success of the
physical examination workshop and the bellringer sessions. These hands-on
sessions significantly increased the students' skill level and retention of
the course material. The enthusiasm of the students for these sessions was
high and ranked above the average enthusiasm for the more traditional
small-group sessions conducted in previous years. This format is, however,
labor-intensive for the faculty. To attain the number of faculty needed for
each session, it was necessary to utilize volunteer clinical as well as
full-time faculty. The volunteer clinical faculty were very willing and were
comfortable teaching from the outlines provided. They were uniformly
enthusiastic, and 100% volunteered to return to teach the course the next
year. The physical examination and bellringer format, in which students
rotated to a new instructor every thirty minutes, provided a more uniform
experience for the students and engendered a level of enthusiasm, on the part
of both students and instructors, that was better than that associated with
the format used in the initial small-group sessions in 2000, in which one
instructor stayed with the same student group for the entire afternoon. The
strategy, in the bellringer sessions, of reiterating and expanding on the
content presented in lectures and in the physical examination workshops worked
well, as indicated by the progressive increase in student confidence in the
material as they progressed through the course.
The crucial first step in initiating change in this course was obtaining
the agreement of the three departments to relinquish their individual course
time and commit to a single integrated course. Integration at the director
level was highly successful and led to thoughtful discussions regarding
selection of the content to be included from each discipline. Better selection
of lecturers, mentoring of lecturers, and implementation of the PowerPoint
format improved the overall quality of the lectures. When changes were needed,
one of the directors worked with the lecturer to modify the content and
graphics. In the second year, all lectures were posted on the course web site
immediately after the lecture. Students appreciated access to the lectures on
the web; however, they wanted a paper lecture handout at the beginning of the
course as well.
Three changes made in the first year were not successful. The four
theater-style multidisciplinary case presentations designed to illustrate the
clinical application of lecture material were rated the lowest of the course
components by the students. The theater-style presentations were abandoned,
and the key content was integrated into the lectures or the small-group
sessions (physical examination workshop or bellringer sessions). The
stand-alone clinical anatomy lectures designed to review material learned in
gross anatomy for use in the clinical setting were also not as effective as we
had hoped. In the second year of the course, we integrated the anatomy review
into the lectures and bellringer sessions. The web-based cases for independent
study had technical problems, with server and browser incompatibilities. The
web-based cases were not used in the second year, while we were solving the
technical problems.
Better education of physicians is essential if they are to successfully
meet the challenge of the increase in musculoskeletal diseases that will
accompany the aging of our population. We believe that this education should
start early. Exposure of second-year medical students to an integrated course
encompassing basic concepts in orthopaedics, rheumatology, and physical
medicine and rehabilitation has increased our students' knowledge and
confidence in this area. The heuristic strategy of progressively introducing
material in lectures and physical examination workshops and reinforcing the
material with small-group discussions proved to be very effective. One
unexpected effect of the changes in the course was the remarkable increase in
enthusiasm among the students as well as the faculty. The students' comments
were remarkable in their enthusiasm and for the appreciation that they
expressed for these opportunities to interact with a diverse group of faculty
in a relaxed but structured environment.
Tables showing the students' levels of confidence and their ratings for
each course element are available with the electronic versions of this
article, on our web site at
(go to the article citation and click on "Supplementary Material")
and on our quarterly CD-ROM (call our subscription department, at
781-449-9780, to order the CD-ROM).
Yelin E, Herrndorf A, Trupin L,
Sonneborn D. A national study of medical care expenditures for
musculoskeletal conditions: the impact of health insurance and managed care.
Arthritis Rheum.2001;44:
1160-9.441160
2001
[PubMed][CrossRef]
Yelin E, Callahan LF. The
economic cost and social and psychological impact of musculoskeletal
conditions. National Arthritis Data Work Groups. Arthritis
Rheum.1995;38:
1351-62.381351
1995
[CrossRef]
Simon LS. Osteoarthritis: a
review. Clin Cornerstone.1999;2:
26-37.226
1999
[PubMed][CrossRef]
Clawson DK, Jackson DW, Ostergaard
DJ. It's past time to reform the musculoskeletal curriculum.
Acad Med.2001;76:
709-10.76709
2001
[PubMed][CrossRef]
Pinney SJ, Regan WD. Educating
medical students about musculoskeletal problems. Are community needs reflected
in the curricula of Canadian medical schools? J Bone Joint Surg
Am.2001;83:
1317-20.831317
2001
Good PI.The design
and contact of clinical trials. New Jersey: Wiley;
2002. p 174.174
2002