Abstract
Background:
Much of the difficulty in understanding acetabular fracture patterns is due to the complex three-dimensional relationship of the acetabulum to the greater pelvis. We hypothesized that combining three-dimensional “hands-on” anatomic models with two-dimensional informational teaching sheets would improve the ability of orthopaedic residents to accurately classify acetabular fracture patterns and aid in preoperative surgical approach selection.
Methods:
Thirty-five orthopaedic residents from two programs accredited by the Accreditation Council for Graduate Medical Education participated in this prospective study. Twenty-question quizzes based on radiographs and computed tomography images of acetabular fractures tested the ability of the residents to accurately classify these fractures. One-half of the residents had access to informational teaching sheets only, and the other group had access to three-dimensional pelvic models of the fractures in addition to the informational sheets.
Results:
There was a positive correlation between the postgraduate year in training and the mean pre-intervention quiz score (r2 = 0.89). The mean improvement in the quiz score was 15% ± 15% for first and second-year residents compared with 3% ± 12% for fourth and fifth-year residents (p = 0.04). The resident group that used the three-dimensional “hands-on” models showed greater post-intervention improvement in the quiz score.
Conclusions:
In this preliminary study, active learning that incorporated three-dimensional ”hands-on” pelvic models improved the ability of orthopaedic residents to accurately classify acetabular fracture patterns compared with use of informational teaching sheets alone.
Fractures of the acetabulum are clinically challenging injuries that can pose substantial diagnostic as well as treatment challenges1,2. Much of the difficulty in understanding acetabular fracture patterns is due to the complex three-dimensional relationship of the acetabulum to the greater pelvis. It is the accurate determination of the fracture pattern that allows for adequate preoperative planning and choice of an appropriate surgical approach, both of which are prerequisites for a good surgical outcome3,4. Judet and Letournel described five elementary and five associated fracture patterns in their seminal text, Les Fractures du Cotyle (Fractures of the Acetabulum), first published in 19745. This classification scheme is the most widely used by orthopaedic surgeons, as it has the best interobserver and intraobserver reliability and it facilitates the choice of surgical approaches and fixation techniques6.
An important component of an orthopaedic resident’s training is proficiency in the diagnosis and treatment of musculoskeletal trauma. Acetabular fracture classification is predicated on accurate interpretation of radiographs and computed tomography (CT) scans. Acetabular fractures are difficult for junior orthopaedic residents to comprehend because of their three-dimensional anatomic complexity, their low incidence1, and the limited opportunities for resident participation in the surgical treatment of these fractures. Historically, the learning of material related to these fractures has been passive, taking the form of either reading texts or listening to presentations by peers or faculty members. In contrast, active, tactile learning may be afforded by utilizing three-dimensional “hands-on” models7. However, this requires the instructor and student to manipulate and interact with the subject matter to produce new insights and perceptions. Three-dimensional anatomic models (e.g., Sawbones; Pacific Research Laboratories, Vashon, Washington) are currently used by orthopaedic residents to learn surgical techniques of fracture fixation as well as to become familiar with new surgical implants. We hypothesized that these models could also facilitate understanding of complex fracture patterns.
To our knowledge, this represents the first study evaluating the use of three-dimensional anatomic models in teaching orthopaedic residents the diagnosis of acetabular fracture patterns and the standard methods of acetabular internal fixation. Our null hypothesis was that active learning that combined two and three-dimensional teaching aids would not significantly improve the ability of orthopaedic residents to accurately classify acetabular fracture patterns or choose an appropriate surgical approach compared with learning that used two-dimensional teaching aids alone.
This study was designed to determine the impact of a teaching module that included a three-dimensional anatomic model on the ability of orthopaedic surgical residents to correctly classify acetabular fracture patterns and to choose an appropriate surgical treatment approach. Recognition of acetabular fracture patterns was assessed by the administration of a series of twenty-question image-based quizzes. The first intervention involved use of informational teaching sheets alone, and the second involved use of three-dimensional pelvic models of individual acetabular fractures in conjunction with the corresponding informational teaching sheets. Pre-intervention and post-intervention quizzes were used to determine the effect of each intervention on accurate recognition of acetabular fracture patterns and appropriate surgical approaches for those individual fractures.
The authors, two of whom are orthopaedic trauma fellowship-trained surgeons, developed the informational sheets, models, and quizzes in conjunction with other orthopaedic trauma fellowship-trained faculty at our institution. The informational sheets incorporated images, diagrams, and images from the Judet textbook, Les Fractures du Cotyle5, and the AO (Arbeitsgemeinschaft für Osteosynthesefragen) web site. The images used in the quizzes were reviewed by multiple faculty members, and unclear images were excluded. The study was approved by our institution’s Committee on Human Research, and all participants gave informed consent.
Sample Population
All of the orthopaedic surgery residents from two local orthopaedic surgery departments (both accredited by the Accreditation Council for Graduate Medical Education [ACGME]) were invited to participate in the study. An initial twenty-question pre-intervention quiz was given to the entire group. The residents were then randomly split into two groups; the distribution of residents according to postgraduate year in training (PGY) was similar in the two groups. One group was asked to study using the informational teaching sheets only, whereas the other group had access to both the informational teaching sheets and the three-dimensional pelvic models. The residents had eight minutes to study each of the fracture pattern modules. As we were limited by the residency directors to a two-hour block of time usually reserved for didactics in the curriculum, this amount of time represented the best compromise to allow for administration of both quizzes and an equal opportunity to study each fracture pattern module. The groups then took a twenty-question post-intervention quiz. Approximately two months later, this teaching exercise was repeated. The two groups were switched, so that the group that previously had access to the informational sheets only now had access to the three-dimensional anatomic models and the informational sheets, and the group that previously also had access to the three-dimensional anatomic models now had access to the informational sheets only.
Three-Dimensional Teaching Models (see Appendix)
Ten standard pelvic models were used for the study. On each pelvic model, one of the classic fracture patterns described by Judet and Letournel was created: posterior wall, posterior column, anterior wall, anterior column, transverse, T-type, anterior column with associated posterior hemitransverse, posterior column with associated posterior wall, transverse with associated posterior wall, and associated both-column fractures. The right acetabulum was cut with an oscillating saw to recreate each fracture pattern. The fracture fragments were displaced as they would be clinically on the basis on the deforming forces and were held in space relative to the pelvic model by multiple threaded Kirschner wires. The same fracture lines were recreated on the contralateral (i.e., left) acetabulum, and plates and screws (Synthes, Paoli, Pennsylvania) were fixed to the model in a manner representing a common fixation construct appropriate for these fractures. Each pelvis was housed in its own clear plastic box together with its corresponding informational sheet. The plastic box was used for organization and to prevent inadvertent damage to the models when they were not in use, but the models were easily removed from the boxes to allow for interaction, manipulation, and study.
Informational Teaching Sheet
An individual informational teaching sheet was created for each of the acetabular fracture patterns. This sheet provided a succinct summary of the salient clinical information regarding the particular fracture type, as well as diagrams and schematics that served as visual aids for learning (Figs. 1-A and 1-B). Standard radiographs that had been made in the routine evaluation of acetabular fractures (including anteroposterior and Judet oblique view radiographs) and representative axial CT images were also included on the sheets to illustrate the individual fracture patterns. These images corresponded to the fracture patterns created on the right acetabulum of the three-dimensional pelvic models. Finally, a series of postoperative radiographs were included on each sheet to reflect the common fixation technique shown on the left hemipelvis of the models. All radiographs and CT scans were initially reviewed by the authors and a group of fellowship-trained orthopaedic trauma surgeons, and any unclear images were excluded. Two separate informational sheets were included to provide additional background information regarding normal radiographic parameters of the pelvis and surgical approaches to the acetabulum. The informational teaching sheets included the information needed to answer all of the quiz questions correctly.
Quizzes
The twenty-question quizzes were given via a PowerPoint (Microsoft, Redmond, Washington) presentation projected onto a screen. Each image of a particular acetabular fracture or fixation method was shown for thirty seconds before advancing to the next slide. Each question included either a series of radiographs only, a series of axial CT scans only, or a combination of both radiographs and CT scans. The residents were asked to determine the unique acetabular fracture pattern depicted and an appropriate approach to reduce and fix the acetabular fracture. When there was more than one possible answer, the residents were asked to choose the one best answer. All questions were scored equally. No question was repeated on the subsequent post-intervention quiz. All of the twenty-question quizzes were designed to be similar with regard to the level of difficulty. This was accomplished by populating the quiz with an equal number of questions on elementary and associated fracture patterns. Overall, four unique quizzes were prepared (see Appendix). The second round of quizzes was administered two months later.
Questions that <30% of the residents answered correctly were omitted from the final statistical analysis. This approach has been validated by other educators who work on test development because such questions do not appear to discriminate between true understanding of the material and guessing7,8. We hypothesized that such questions may also have included images that were unclear or difficult to interpret. Specifically, during the first round of testing, five questions were omitted from the pre-intervention quiz and one question was omitted from the post-intervention quiz. During the second round of testing, two questions were omitted from the pre-intervention quiz and four questions were omitted from the post-intervention quiz. Quiz scores were reported as the percentage of the total number of scored questions that were answered correctly.
Statistical Analysis
Descriptive statistics consisted of the mean, median, standard deviation, and range of the pre-intervention and post-intervention quiz scores. A paired Student t test was used to compare scores between groups of residents with regard to the effect of the intervention. Linear regression analysis was performed to analyze the correlation between quiz scores and resident PGY. A correlation coefficient of r > 0.7 (i.e., r2 > 0.5) was considered to indicate a strong positive correlation.
Source of Funding
This project was funded by an Orthopaedic Trauma Association Resident Research Grant (E.H.), and all hardware was donated by Synthes USA.
Thirty-five (83%) of the forty-two orthopaedic residents in the two training programs were able to participate in at least one of the two teaching sessions. Five were in PGY 1, eight in PGY 2, nine in PGY 3, six in PGY 4, and seven in PGY 5.
There was a strong positive correlation between the resident PGY and both the pre-intervention quiz score (r2 = 0.89) and the post-intervention quiz score (r2 = 0.57). The mean pre-intervention quiz score for the residents in PGYs 1 and 2 was 46% ± 13% compared with 67% ± 17% for PGYs 4 and 5 (p = 0.0018). For the post-intervention quizzes, residents in PGYs 1 and 2 scored 60% ± 17% compared with 70% ± 16% for PGYs 4 and 5 (p = 0.142). The post-intervention improvement in quiz score was also related to the PGY. The mean improvement in the quiz score was 15% ± 15% for PGYs 1 and 2 compared with only 3% ± 12% for PGYs 4 and 5; this difference was significant (p = 0.04) (Fig. 2). Similar results were seen during the second testing round.
The mean pre-intervention quiz score was similar in the group using pelvic models (56% ± 20%) and the group using informational teaching sheets only (56% ± 16%, p = 0.98). The three-dimensional model group achieved a mean post-intervention score of 71.2% ± 17%, which was significantly higher than their pre-intervention score (p = 0.00061). The informational sheet group achieved a lower mean post-intervention score of 61% ± 14%, which not still significantly higher than their pre-intervention score (p = 0.13). The relative improvement in the pelvic model group was significantly greater than the improvement in the informational sheet group (15% ± 15% compared with 5% ± 11%, p = 0.03) (Fig. 3).
The gain in knowledge in the pelvic model group relative to the information sheet group was not retained during the interval between testing rounds. In the second testing round, when the interventions were switched, the two groups again had similar mean pre-intervention quiz scores (47% ± 20% compared with 47% ± 21%, p = 0.973). The new pelvic model group (old informational sheet group) had a significant 16% ± 17% improvement from their pre-intervention quiz score to a post-intervention score of 63% ± 18% (p = 0.012). Similarly, the new informational sheet group (old pelvic model group) had a significant 7% ± 14% improvement from their pre-intervention score to a post-intervention score of 54% ± 14% (p = 0.049). Neither the difference between the mean post-intervention quiz scores nor the difference in improvement between the groups reached significance (p = 0.174 and p = 0.144, respectively).
We also performed a secondary analysis focusing on the proportion of residents who would obtain a “passing” score of 75%. In the three-dimensional model group, 4% of residents had a passing grade before the intervention and 43% had a passing grade after the intervention (an increase of 39%, p = 0.0002). In the group given informational sheets only, the passing percentage increased from 3% to 15% post-intervention (+12%, p = 0.04). Although both interventions resulted in a significant increase in the proportion of residents who “passed” the test, the addition of the three-dimensional model resulted in a significantly greater increase in the passing rate compared with use of the informational sheets alone (39% compared with 12%, p = 0.02).
In the secondary analysis, learning by senior residents did not demonstrate a “ceiling effect.” The percentage of PGY-1 and 2 residents with a passing grade increased from 0% to 25% (+25%) after the first round of interventions and from 0% to 16% (+16%) after the second round. A similar increase was seen in the PGY-4 and 5 residents, with the percentage with a passing grade increasing from 8% to 33% (+25%) in the first round and from 10% to 35% (+25%) in the second round. Both of these groups of residents benefited more from using the three-dimensional models than from using the informational sheets alone, with the percentage of PGY-1 and 2 residents who passed increasing from 0% to 36% (+36%, p = 0.04) in the model group compared with 0% to 7% (+7%, p = 0.34) in the informational sheet group, and the percentage of PGY-4 and 5 residents who passed increasing from 13% to 50% (+37%, p = 0.38) in the model group compared with 9% to 18% (+9%, p = 0.34) in the informational sheet group.
This study confirmed that active learning incorporating three-dimensional pelvic models improved the ability of orthopaedic residents to learn the classification of acetabular fracture patterns compared with reliance on only two-dimensional informational sheets. The knowledge acquisition was, however, only short-term, necessitating repeated interventions. The concept of utilizing a “working model” has previously been demonstrated to be similarly effective for teaching the complex functional and applied anatomy of the subtalar joint9. In that study, the orthopaedic residents who were taught using the “working model” of the talus and calcaneus not only performed better on a post-intervention multiple-choice quiz than those taught using written texts and diagrams, but they also felt subjectively more confident in their understanding of the anatomy.
Orthopaedic surgery residents acquire a progressively better understanding and recognition of acetabular fracture and fixation patterns during their training. In this study, the residents in their last two years of training answered approximately two-thirds of the pre-intervention questions correctly, whereas those in their first two years of training answered slightly less than 50% correctly. This suggests that both didactic teaching of the material and cumulative exposure to the subject during residency improve understanding of the subject.
The mean quiz score of the residents in each PGY improved after either intervention. As none of the questions on the post-intervention quiz were repeated from the initial quiz, this indicates the acquisition of genuine understanding, rather than pattern recognition, on the part of the residents. Not surprisingly, the more junior residents had the greatest improvement in their scores after the teaching module, whereas the most experienced, fifth-year, residents had the least improvement. In a previous study in the general surgery literature that focused specifically on evaluating technical surgical skills, a similar “ceiling effect” in the third and fourth years of training was noted10. The authors concluded that either assessment tools for evaluating technical performance need to be more sensitive or the evaluation of senior trainees should involve other aspects of surgical competence. It is interesting to note that this “ceiling effect” was not seen in our secondary analysis focusing on the proportion of residents who would obtain a “passing” score of 75%.
The addition of the three-dimensional anatomic model to the teaching module resulted in a significant additional improvement in the post-intervention quiz score. After only ninety minutes of study, residents who had access to both elements of the teaching module had a 15% improvement in their quiz score compared with a 5% improvement without the models. Moreover, the model intervention had the greatest effect on the more junior residents. During the first round of quizzes, the first-year residents in the model group improved their scores by 37%, and during the second round of this teaching exercise, the second-year residents improved their scores by 36%. The improved comprehension and resultant improved quiz scores may be the result of the active, three-dimensional tactile learning afforded by the models. In recent years, this type of motor learning has been investigated as it relates to teaching complex anatomy9 as well as to the acquisition of technical surgical skills through the use of both bench models and simulators11.
Similar results were obtained when the second session of the teaching exercise was conducted two months after the original intervention. Again, there was a strong correlation between the pre-intervention quiz score and resident PGY, and the greatest effect of the modules was realized by the more junior residents. A somewhat unexpected finding was that the pre-intervention quiz scores at the onset of the second round did not differ significantly between the two intervention groups. We had initially hypothesized that the group who had access to the pelvic models during the first teaching exercise would also retain their knowledge better than the informational sheet group. One possible explanation for this lack of difference is that the time allotted with the models was too short to result in long-term retention. These results suggest that the models should be available for resident teaching for longer periods of time and/or for repeated study periods throughout the residency years.
Although the results of this initial pilot study are promising, it has several important limitations. First, we tested this module only on the residents in two ACGME-accredited programs, and the applicability of the findings to other orthopaedic resident trainees is unknown. Additionally, we did not validate the quizzes and no psychometric analyses were applied to the questions that were used. Although efforts were made to adjust the pre-intervention and post-intervention quizzes to be comparable in difficulty, by including an equivalent number of questions on elementary and associated acetabular fractures and by eliminating questions that were not answered correctly by at least 30% of the residents, there may have been subtle differences that affected the magnitude of the measured improvement in learning. Moreover, independent study by the residents between the research sessions could not be controlled and may have affected the results.
We were able to show a significant added benefit of the three-dimensional anatomic models compared with the informational teaching sheets alone. Moreover, this technique appeared to be especially beneficial to residents in their first three years of training. The constrained time and environment in which the models were used during this investigation may have resulted in an underestimation of their true potential. In fact, the real benefit of this approach to learning may only be fully realized when trainees have unlimited time to use the three-dimensional “hands-on” models and informational teaching sheets in the context of evaluating images of a real patient’s injury. Finally, we believe that it may be possible to translate this concept of a “hands-on” teaching model to other areas of orthopaedics that involve three-dimensionally complex anatomy, with a similarly beneficial effect on resident education.
Presentations showing the acetabular fracture models and the quizzes are available with the online version of this article as a data supplement at jbjs.org.
Laird
A;
Keating
JF. Acetabular fractures: a 16-year prospective epidemiological study. J Bone Joint Surg Br.
2005 Jul;87(
7):969-73.[CrossRef]
Tornetta
P 3rd. Displaced acetabular fractures: indications for operative and nonoperative management. J Am Acad Orthop Surg.
2001 Jan-Feb;9(
1):18-28.
Baumgaertel
F. [Diagnosis, classification and surgical indications in acetabulum fractures]. Orthopade.
1992 Nov;21(
6):427-41. .
Matta
JM;
Merritt
PO. Displaced acetabular fractures. Clin Orthop Relat Res.
1988 May;(
230):83-97.
Judet
R;
Letournel
E. Les fractures du cotyle. Paris: Masson; 1974.
Beaulé
PE;
Dorey
FJ;
Matta
JM. Letournel classification for acetabular fractures. Assessment of interobserver and intraobserver reliability. J Bone Joint Surg Am.
2003 Sep;85-A(
9):1704-9.
Gronlund
NE. How to construct achievement tests. 4th ed.Englewood Cliffs, NJ: Prentice Hall; 1988.
Welsh
AL. Multiple choice objective tests. : Saunders
P;
Welsh
AL;
Hansen
WL, . Resource manual for teaching training programs in economics. New York: Joint Council on Economic Education; 1978. 191-228.
Joseph
B;
Rebello
G;
Mayya
S. The use of a working model for teaching functional and applied surgical anatomy of the subtalar joint. Foot Ankle Int.
2006 Apr;27(
4):286-92.
Munz
Y;
Moorthy
K;
Bann
S;
Shah
J;
Ivanova
S;
Darzi
SA. Ceiling effect in technical skills of surgical residents. Am J Surg.
2004 Sep;188(
3):294-300.[CrossRef]
Hamstra
SJ;
Dubrowski
A;
Backstein
D. Teaching technical skills to surgical residents: a survey of empirical research. Clin Orthop Relat Res.
2006 Aug;449:108-15.
Disclosure: One or more of the authors received payments or services, either directly or indirectly (i.e., via his or her institution), from a third party in support of an aspect of this work. In addition, one or more of the authors, or his or her institution, has had a financial relationship, in the thirty-six months prior to submission of this work, with an entity in the biomedical arena that could be perceived to influence or have the potential to influence what is written in this work. No author has had any other relationships, or has engaged in any other activities, that could be perceived to influence or have the potential to influence what is written in this work. The complete Disclosures of Potential Conflicts of Interest submitted by authors are always provided with the online version of the article.