Abstract
Background:
The visual overlay technique for surgical planning is difficult to apply to spatially complex fractures. Virtual reality can be applied by virtual fracture carving to adapt the visual overlay technique to three-dimensional (3D) images. In this study, we evaluated whether virtual fracture carving is a useful exercise by performing two experiments comparing trainees’ understanding of a complex fracture with the application of either current preoperative techniques or the use of the Virtual-Fracture-Carving Simulator.
Methods:
Forty-eight participants—senior medical students and residents in postgraduate year 1 (PGY1)—were asked to learn the anatomy of an associated both-column acetabular fracture. The participants were randomized into three groups: control, Sawbones, and virtual (Virtual-Fracture-Carving Simulator). The randomization protocol was stratified for sex and visuospatial ability. The measure of learning was a fracture line-drawing task evaluated for nineteen anatomic relationships.
Results:
The virtual group performed better than both the control and the Sawbones group, with an absolute difference in score of 22.7% (p = 0.0001) and 17.8% (p = 0.0026), respectively. There was no significant difference between the control and Sawbones groups. The virtual group drew fracture characteristics requiring a higher level of spatial understanding with greater accuracy.
Conclusions:
The results of this study validate the concept behind the visual overlay planning technique—i.e., that thoughtful play promotes understanding of fracture anatomy. These results objectively demonstrate that the use of a Virtual-Fracture-Carving Simulator is feasible, and superior to conventional preoperative planning strategies in terms of quantity and quality of understanding of a spatially complex fracture.
To plan for the surgical treatment of a fracture, orthopaedic residents are taught to perform preoperative templating using the “visual overlay” technique. This exercise consists of three steps: tracing fracture fragments on radiographs, reducing the fracture by “solving the puzzle,” and determining the placement of fracture fixation1. Working through these steps helps the trainee to better understand anatomy and rehearse surgical steps, ultimately facilitating a safe and efficient surgical procedure. However, the visual overlay templating technique is not well suited to anatomically complex areas, such as the acetabulum, which are difficult to visualize on radiographs1,2.
Even Letournel and Judet conceded their difficulty understanding acetabular fractures after four years of focused study2. Despite their contribution to the understanding of radiographs and surgical management of acetabular fractures, these complex injuries continue to challenge orthopaedic surgeons3-5. Some have argued that one must “gain experience for a time directly with an experienced surgeon who, through operative experience and radiographic interpretation, can directly relate radiographic anatomy to [acetabular] bone anatomy.” Although frequently used, computed axial tomography (CT)—including three-dimensional (3D) reconstructions—has proven beneficial only for identifying loose bodies and fracture displacement and impaction, not for visualizing global fracture anatomy4,6-9. Although 3D CT reconstructions are increasingly being used, their role has not yet been clarified10-12. Letournel and Judet recommended that surgeons draw fractures onto a Sawbones model preoperatively2, but the benefit of this exercise has also not been investigated, to our knowledge. Effective templating prior to acetabular fracture surgery would be beneficial. Virtual reality could potentially be used for this purpose.
Virtual reality refers to computer-generated environments that are designed to approximate reality. The virtual reality environment can be created with use of a variety of interfaces13. A potentially useful virtual reality interface for fracture surgery is haptics, which refers to devices that allow the user to feel resistance, contours, textures, and edges of objects displayed on a computer monitor, replicating the sense of touch14.
A previous study demonstrated that rehearsing surgical steps on a virtual reality haptic interface is a useful exercise for planning acetabular fracture surgery15. The current study was designed to assess the feasibility of applying virtual reality technology to enhance the cognitive aspects of preoperative planning for fracture surgery. To accomplish this, we developed a Virtual-Fracture-Carving Simulator using a commercially available haptic virtual reality device and a 3D CT reconstruction of an intact pelvis. The concept guiding the Virtual-Fracture-Carving Simulator was to recreate the first step of the visual overlay technique—i.e., tracing—in 3D. The primary outcome of this feasibility study was comprehension of fracture anatomy, not surgical performance.
Our primary research question addressed the usefulness of the Virtual-Fracture-Carving Simulator in two experiments. The first experiment tested the null hypothesis that there would be no significant difference in the understanding of a complex fracture between trainees preparing with current preoperative preparation techniques (viewing a 3D CT scan alone, and viewing a 3D CT scan and tracing fracture lines onto a Sawbones model). The second experiment tested the null hypothesis that there would be no significant difference in understanding between trainees preparing with the Virtual-Fracture-Carving Simulator and those preparing with the intervention identified as superior in our first experiment. The secondary research question addressed the null hypothesis that inherent visuospatial ability would not modify the effect of the Virtual-Fracture-Carving Simulator intervention.
Study Design
A prospective, randomized, three-arm study was designed, and research ethics board approval was obtained. A power analysis for the main effect of a learning intervention (minimum significant absolute difference in outcome score = 15%, alpha = 0.05, and beta = 0.2), based on the results of a pilot study, indicated a minimum of sixteen participants would be required in each of the three groups. Medical school and residency training administrators e-mailed and posted recruitment letters, asking interested individuals to contact the investigators. All individuals who responded to recruitment efforts agreed to participate and understood that there would be no repercussions for dropping out of the study. Forty-eight individuals, who were either non-orthopaedic residents in postgraduate year 1 (PGY1) or senior medical students without substantial knowledge of acetabular fractures, were recruited to participate.
A stratified-block randomization protocol was used to allocate participants to three learning groups: control, Sawbones, and virtual (Virtual-Fracture-Carving Simulator). Participants were first stratified for sex16,17 and visuospatial ability to balance the distribution of these suspected confounders. High and low strata for visuospatial ability were defined by the median Mental Rotations Test Part-A score previously determined for medical students18-20. Within each stratum, randomization then proceeded in permuted blocks (a block size of nine, and an allocation ratio of 1:1:1), generated with use of a random-number generator.
The participants were asked to learn the fracture lines of an associated both-column acetabular fracture. The exact cognitive skills evaluated in the test of understanding, and how they related to the anatomy of an associated both-column fracture, were outlined to the participants with an emphasis on how these skills should be developed when learning the associated both-column fracture. A twenty-minute learning time limit was set for all groups, and no other form of feedback was provided. This time limit was based on our impression that the average resident would spend twenty minutes preparing for an acetabular fracture case and that this time was sufficient to learn the anatomy required for the fracture-drawing task.
Control Group
The participants were given a 3D CT reconstruction of a left hemipelvis with an associated both-column fracture and an excerpt outlining the defining anatomic characteristics of the associated both-column fracture from the textbook by Letournel and Judet2.
To create the 3D CT reconstruction, a DICOM (Digital Imaging and Communications in Medicine) file of a CT scan of a normal pelvis was imported into Magics software (Materialise, Leuven, Belgium). The file was converted to the STL (stereolithography) format, and only the left hemipelvis was retained. To create an intact-hemipelvis template for haptic software, the STL file was then imported into a desktop computer running FreeForm Modeling version-10 software interfaced with a PHANTOM Desktop haptic device (SensAble Technologies, Woburn, Massachusetts) (Fig. 1). A distinct fracture was carved into this intact-hemipelvis template with use of the haptic device, but fragments were not displaced. The carved-hemipelvis template was then exported back to Magics software to generate the 3D CT reconstruction. Participants viewed the 3D CT reconstruction in the Magics-software environment, enabling them to magnify and fully rotate the 3D CT reconstruction along all axes.
Sawbones Group
In addition to a 3D CT reconstruction of the carved hemipelvis and textbook excerpt, the participants were given a model of a left hemipelvis. Through a standardized script, suggestions were made on how to use the Sawbones model, but the suggestions were not enforced. Palpation of the landmarks discussed in the textbook and drawing fracture lines on the model were encouraged.
Virtual Group
In addition to a 3D CT reconstruction of the carved hemipelvis and textbook excerpt, participants were given a Virtual-Fracture-Carving Simulator. A standard five-minute task, unrelated to acetabular fractures or pelvic anatomy, was given to participants in this group so that they could become familiar with the interface. The participants were then asked to carve fracture lines into the virtual reality model of the intact hemipelvis and to frequently compare their lines against the 3D CT reconstruction of the carved hemipelvis.
The Virtual-Fracture-Carving Simulator was developed with use of commercially available hardware and software. A desktop computer running FreeForm Modeling version-10 software interfaced with a PHANTOM Desktop haptic device (SensAble Technologies) (Fig. 1). The haptic device is held like a pen and controls a stylus displayed on the computer monitor. The stylus can be used to palpate anatomic landmarks on the 3D CT reconstruction of an intact pelvis, and to carve fracture lines (see Appendix [Video 1]). The participants carved fracture lines into the same intact-hemipelvis template for haptic software as was utilized to create the 3D CT reconstruction of the carved hemipelvis.
Test of Understanding
Experienced orthopaedic trauma surgeons were asked to identify how knowledge of fracture anatomy should be translated to the operating room. They identified four key cognitive skills: (1) recognition of landmarks by visual cues, (2) recognition of landmarks by tactile cues, (3) understanding the relationship between landmarks and fracture lines, and (4) mentally rotating the mental schema of landmarks and fracture lines to the patient’s position. A fracture-drawing task was designed to test these cognitive skills.
For the test, the participants from all groups were asked to draw fracture lines onto the right hemipelvis of a complete Sawbones pelvis model (both innominate bones and the sacrum). The model was positioned prone, and the participant approached the cranial end. The anatomy inferior to the acetabular roof was draped, and not visualized, but was accessible by touch. Participants were given five minutes to complete the task. The fracture lines were scored with respect to nineteen anatomic characteristics of an associated both-column fracture derived from the description of the associated both-column fracture by Letournel and Judet2. One point was given for each correctly drawn feature, and the score was the sum of all points.
Data Collection
The participants’ year in training and sex were recorded. The Mental Rotations Test Part A was used to assess inherent visuospatial ability18,19. This test assesses an individual’s ability to recognize complex 3D structures from different viewpoints, and therefore is used to evaluate high-level visual and spatial skills.
Statistical Analysis
A three-way analysis of variance (ANOVA) with an interaction term for learning intervention and visuospatial ability was performed. The first two hypotheses were evaluated with use of contrasts. The third was evaluated with use of the F-test for the interaction term. The Bonferroni correction was used to keep the overall Type-I error rate at 5% for the three tests (α = 0.016 for each test). Post hoc comparisons of intervention means were performed with use of the Tukey honestly significant difference.
Source of Funding
An internal university grant (“Educational Initiatives in Residency Education” grant from the office of Postgraduate Medical Education at the University of Ottawa) funded the purchase of the PHANTOM Desktop haptic device. The funding source had no role in the study.
The distribution of demographic variables was balanced among the three groups. As a stratified randomized design was utilized, the distribution of sex and visuospatial ability was identical in the three learning groups. The level of training was not controlled during the allocation stage, but randomization was effective as the three learning groups were balanced in terms of the distribution of third and fourth-year medical students and PGY1 residents.
There was no significant difference between the performances of the control and Sawbones groups (p = 0.3656). However, the virtual group performed significantly better than the pooled control and Sawbones group (p < 0.0001). No evidence of effect modification was found as the interaction between learning intervention and visuospatial ability was not significant (p = 0.8859).
Pairwise comparisons of means for the main effect of learning group, controlled for other covariates, indicated that the virtual group performed better than both the control and the Sawbones group, with an absolute difference in score of 22.7% (p = 0.0001) and 17.8% (p = 0.0026), respectively (Fig. 2).
To discern differences between groups, the performance with respect to each fracture characteristic evaluated in the fracture-drawing score was plotted for the virtual group and the pooled control/Sawbones group (Fig. 3). Marked differences between groups were seen for five characteristics. The virtual group drew concordant lines of the anterior-column component on the inner and outer aspects of the pelvis more frequently. The remainder of the differences occurred on the inner table of the pelvis. The virtual group was more likely to draw the anterior and posterior-column components traversing and intersecting on the quadrilateral plate. Additionally, the virtual group was more likely to draw, on the inner aspect of the pelvis, the posterior-column component of the fracture line beginning at the summit of the greater sciatic notch. However, there was no difference with regard to drawing the fracture line originating from the greater sciatic notch on the exterior aspect of the pelvis; all groups performed poorly.
The results of this study demonstrate that it is feasible for inexperienced trainees to learn fracture anatomy effectively in virtual reality. Participants who learned using the Virtual-Fracture-Carving Simulator drew the fracture lines with higher accuracy and were better able to maintain correct fracture spatial relationships between the inner and outer tables of the pelvis than participants using conventional preparatory aids. The benefit of the Virtual-Fracture-Carving Simulator did not depend on inherent visuospatial ability, which is advantageous as residents are not selected on the basis of their innate learner skills.
The lack of preparatory aids for complex fracture surgery motivated us to perform this study. We sought to develop a virtual reality aid based on the gold-standard visual overlay technique for simpler fractures described by Mast et al.1, to facilitate use of the same steps in 3D. In this feasibility study of virtual reality technology, we developed a module for learning only fracture anatomy, not reduction or fixation. To our knowledge, this is the first study formally evaluating preoperative planning techniques in orthopaedic surgery.
Learning outcomes can be cognitive, skill-based, or affective20. To date, most orthopaedic education research has evaluated skill-based outcomes through arthroscopic simulation21. However, it has been recognized that cognition, not psychomotor skills, differentiates master surgeons from novice surgeons22,23, and that cognitive preparation is essential for a successful procedure24. For this study, we were required to develop a new cognitive measure for fracture surgery.
Cognition progresses through three stages of expertise, analogous to the Fitts and Posner model of motor-skill acquisition20,25,26. Kraiger et al. described an initial declarative stage whereby the learner recalls and recognizes facts, a subsequent integrative stage in which knowledge is organized and integrated, and a final strategies phase in which learners apply knowledge efficiently and effectively20. It is recommended that higher-order cognitive stages be measured on the basis of high-level understanding, rather than simple recall and recognition. In consultation with expert trauma surgeons, we identified the high-level understanding of fracture anatomy required intraoperatively.
The results of this study can be explained by considering educational theory. First, it is important to recognize that media such as educational games, video, or virtual reality do not cause learning. Media are only the vehicles that transmit the instructional methods and content, the active learning “ingredients.”27-29 Second, there is no evidence that new media, such as virtual reality, are more motivating than traditional media27-29. Thus, the differences between the conventional and Sawbones groups and the virtual group are likely a result of the different instructional methods and content that the modalities facilitate.
Instructional content refers to the information that is to be learned29. To understand the fracture, participants were required to learn: (1) the appearance of anatomic landmarks, (2) the feel of anatomic landmarks, (3) the orientation of the fracture lines to the landmarks, and (4) the relationship of the fracture lines on the inner and outer aspects of the pelvis. Clearly, the conventional group was at a disadvantage by not being given tactile information by the textbook excerpt and 3D CT reconstruction. Otherwise, all three groups received the same information on visual cues and relationships. As the fracture-drawing test depended heavily on tactile cues, this is a plausible explanation for the difference between the conventional and Sawbones groups and the virtual group.
Instructional methods refer to the structure and techniques with which information is delivered to promote learning29. Beyond providing key information, tactile input is crucial to effectively learn a 3D structure. Recent work has demonstrated that combined visual and haptic sensory input is required to develop a view-independent understanding of a 3D object30,31. View independence refers to the ability to recognize and recall an object from different perspectives. Clearly, such an understanding was necessary to translate fracture anatomy from the learning aids (right hemipelvis, supine, head up) to the fracture-drawing task (left hemipelvis, prone, head down). Again, the Sawbones and virtual groups shared this benefit, suggesting that there must be another reason for the difference between the two groups.
In the education literature, there is debate about the various instructional theories and their application to instructional methods. Merrill suggested an evidence-based cycle of instruction theory in which the learner observes a demonstration, then applies new knowledge in a real-world task, and then is required to draw on existing knowledge and integrate the new knowledge32. In our study, no group was required to draw on existing knowledge. All three groups viewed a demonstration (textbook and 3D CT). The group that received conventional training, without a 3D model, did not have the same opportunity to apply this new knowledge, whereas the Sawbones and virtual groups did have an opportunity to apply new knowledge, perhaps accounting for the improved performance.
During the application-of-knowledge phase, it was much easier for the participant to “undo the fracture carving” on the virtual simulator and recarve it than it was for the participant in the Sawbones group to erase the drawn fracture line and redraw. The ability to “undo and redo” the carving simplified experimentation, allowing participants to more easily challenge and reconstruct their understanding, which likely aided in the integration of the new knowledge. The Virtual-Fracture-Carving Simulator required the participant to correctly carve fracture lines into the inner and outer cortices simultaneously; the user carved the pelvis using a “virtual jigsaw” that cut both cortices at the same time. If the trajectory of the carving was incorrect, the fracture lines on both tables of the pelvis did not correspond. Considering the difference in the nature of the application phase between the learning groups, it is not surprising that the participants in the virtual group were better able to draw fracture lines on the inner aspect of the pelvis, which was entirely hidden from view and had few distinguishing palpable landmarks.
Although this study focused on a single acetabular fracture, it was chosen to provide a generic, unfamiliar complex fracture to be learned by an inexperienced cohort of study subjects. It is reasonable to expect the findings to be generalizable to any population, trainees or surgeons, unfamiliar with any complex fracture. The comparator interventions were realistic and commonly used in practice; as a result, the Virtual-Fracture-Carving Simulator carving exercise is an improvement over what is currently done.
Since virtual reality in itself does not cause learning, it is possible that the learning benefits could be achieved with use of a different module. For example, allowing participants to actually perform an osteotomy on a Sawbones model should produce results equivalent to those achieved with the Virtual-Fracture-Carving Simulator, but this is not practical. First, learners would require several Sawbones models to allow experimentation and to correct their mistakes. Second, to scale up the technique to allow “fracture” reduction and fixation, it would be necessary to provide a full set of operative equipment, as compared with the computer and haptic stylus used in this and other studies15. Another alternative might be the patient-specific 3D models that several trauma equipment manufacturers produce by rapid prototyping from CT scan data. However, those models do not permit the user to experiment, which we believe is the main benefit of the Virtual-Fracture-Carving Simulator. Thus, the virtual reality interface is a versatile cost-effective medium that facilitates high-level learning exercises.
A limitation of this study is that non-orthopaedic trainees were recruited and tested without typical tactile and visual cues apparent in a normal surgical dissection. In this proof-of-concept study, unspecialized junior trainees were recruited to achieve an adequate sample size, with a uniform minimal baseline understanding of acetabular fractures. Our data suggest that this tool would likely be, at the very least, a useful tool for the novice orthopaedic trainee. Additionally, this study did not test long-term retention, only short-term recall. As the Virtual-Fracture-Carving Simulator is geared for preoperative planning, it was reasonable to only evaluate recall. As this was a feasibility study, we only attempted to recreate the first tracing step in the visual overlay preoperative templating technique.
To our knowledge, this is the first study to quantitatively evaluate the effects of preoperative planning in fracture surgery. We identified only one study in the orthopaedic literature evaluating any type of cognitive surgical skill training; it demonstrated a positive effect on performance33. It is apparent that many of the key surgical cognitive skills34—mental readiness, flexible decision-making, forward planning, and awareness of potential problems—could be developed through use of exercises such as the visual overlay technique, or with the Virtual-Fracture-Carving Simulator. Thus, from a research perspective, fracture surgery is an ideal model in which to validate the assertion that cognitive rather than psychomotor skills define competence. From a clinical perspective, sophisticated cognitive preparation can readily be undertaken for fracture surgery with use of available technology.
In conclusion, this study validates the concept behind the visual overlay planning technique that thoughtful practice promotes understanding of fracture anatomy. Furthermore, this study objectively demonstrates that use of a Virtual-Fracture-Carving Simulator is feasible and superior to conventional preoperative planning strategies in terms of quantity and quality of understanding. A fully functional and flexible Virtual-Fracture-Planning Simulator may provide a useful tool with which surgical trainees and established surgeons could plan any operative fracture case.
A video demonstrating a participant carving with the carving tool of the Virtual-Fracture-Carving Simulator is available with the online version of this article as a data supplement at jbjs.org.
Note: The authors thank Oleh Antonyshyn, MD, for helpful discussions, and for sharing haptic equipment, which facilitated the pilot study. They also thank Dr. Tim Ramsay, PhD, for statistical advice.
Mast
J;
Jakob
R;
Ganz
R. Planning and reduction technique in fracture surgery. Berlin: Springer-Verlag; 1989.
Letournel
É;
Judet
R. Fractures of the acetabulum. Elson
RAT, , translator. 2nd ed. Berlin: Springer-Verlag; 1993.
Petrisor
BA;
Bhandari
M;
Orr
RD;
Mandel
S;
Kwok
DC;
Schemitsch
EH. Improving reliability in the classification of fractures of the acetabulum. Arch Orthop Trauma Surg.
2003 Jun;123(
5):228-33. .
Beaulé
PE;
Dorey
FJ;
Matta
JM. Letournel classification for acetabular fractures. Assessment of interobserver and intraobserver reliability. J Bone Joint Surg Am.
2003 Sep;85-A(
9):1704-9.
Visutipol
B;
Chobtangsin
P;
Ketmalasiri
B;
Pattarabanjird
N;
Varodompun
N. Evaluation of Letournel and Judet classification of acetabular fracture with plain radiographs and three-dimensional computerized tomographic scan. J Orthop Surg (Hong Kong).
2000 Jun;8(
1):33-37.
Guy
RL;
Butler-Manuel
PA;
Holder
P;
Brueton
RN. The role of 3D CT in the assessment of acetabular fractures. Br J Radiol.
1992 May;65(
773):384-9.[CrossRef]
Albrechtsen
J;
Hede
J;
Jurik
AG. Pelvic fractures. Assessment by conventional radiography and CT. Acta Radiol.
1994 Sep;35(
5):420-5.
Harley
JD;
Mack
LA;
Winquist
RA. CT of acetabular fractures: comparison with conventional radiography. AJR Am J Roentgenol.
1982 Mar;138(
3):413-7.
Vas
WG;
Wolverson
MK;
Sundaram
M;
Heiberg
E;
Pilla
T;
Shields
JB;
Crepps
L. The role of computed tomography in pelvic fractures. J Comput Assist Tomogr.
1982 Aug;6(
4):796-801.[CrossRef]
Guy
RL;
Butler-Manuel
PA;
Holder
P;
Brueton
RN. The role of 3D CT in the assessment of acetabular fractures. Br J Radiol.
1992 May;65(
773):384-9.[CrossRef]
Martinez
CR;
Di Pasquale
TG;
Helfet
DL;
Graham
AW;
Sanders
RW;
Ray
LD. Evaluation of acetabular fractures with two- and three-dimensional CT. Radiographics.
1992 Mar;12(
2):227-42.
Haveri
M;
Junila
J;
Suramo
I;
Lähde
S. Multiplanar and 3D CT of acetabular fractures. Acta Radiol.
1998 May;39(
3):257-64.
Székely
G;
Satava
RM. Virtual reality in medicine. Interview by Judy Jones. BMJ.
1999 Nov 13;319(
7220):1305.[CrossRef]
Minogue
J;
Jones
MG. Haptics in education: exploring an untapped sensory modality. Rev Educ Res.
2006;76(
3):317-48.[CrossRef]
Fornaro
J;
Keel
M;
Harders
M;
Marincek
B;
Székely
G;
Frauenfelder
T. An interactive surgical planning tool for acetabular fractures: initial results. J Orthop Surg Res.
2010 Aug 4;5:50.[CrossRef]
Voyer
D;
Voyer
S;
Bryden
MP. Magnitude of sex differences in spatial abilities: a meta-analysis and consideration of critical variables. Psychol Bull.
1995 Mar;117(
2):250-70.[CrossRef]
Voyer
D;
Saunders
KA. Gender differences on the mental rotations test: a factor analysis. Acta Psychol (Amst).
2004 Sep;117(
1):79-94.[CrossRef]
Vandenberg
SG;
Kuse
AR. Mental rotations, a group test of three-dimensional spatial visualization. Percept Mot Skills.
1978 Oct;47(
2):599-604.[CrossRef]
Peters
M;
Laeng
B;
Latham
K;
Jackson
M;
Zaiyouna
R;
Richardson
C. A redrawn Vandenberg and Kuse mental rotations test: different versions and factors that affect performance. Brain Cogn.
1995 Jun;28(
1):39-58.[CrossRef]
Kraiger
K;
Ford
JK;
Salas
E. Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. J Appl Psychol.
1993;78(
2):311-28.[CrossRef]
Michelson
JD. Simulation in orthopaedic education: an overview of theory and practice. J Bone Joint Surg Am.
2006 Jun;88(
6):1405-11.[CrossRef]
Cuschieri
A;
Francis
N;
Crosby
J;
Hanna
GB. What do master surgeons think of surgical competence and revalidation?Am J Surg.
2001 Aug;182(
2):110-6.[CrossRef]
McDonald
J;
Orlick
T;
Letts
M. Mental readiness in surgeons and its links to performance excellence in surgery. J Pediatr Orthop.
1995 Sep-Oct;15(
5):691-7.[CrossRef]
Carthey
J;
de Leval
MR;
Wright
DJ;
Farewell
VT;
Reason
JT. Behavioural markers of surgical excellence. Safety Sci.
2003;41(
5):409-25.[CrossRef]
Armstrong
SJ;
Fukami
C. Self-assessment of knowledge: a cognitive learning or affective measure? Perspectives from the management learning and education community. Acad Manage Learning Educ.
2010;9(
2):335-42.[CrossRef]
Reznick
RK;
MacRae
H. Teaching surgical skills—changes in the wind. N Engl J Med.
2006 Dec 21;355(
25):2664-9.[CrossRef]
Bernard
RM;
Abrami
PC;
Lou
Y;
Borokhovski
E;
Wade
A;
Wozney
L;
Wallet
PA;
Fiset
M;
Huang
B. How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Rev Educ Res.
2004;74(
3):379-439.[CrossRef]
Sitzmann
T;
Kraiger
K;
Stewart
D;
Wisher
R. The comparative effectiveness of web-based and classroom instruction: a meta-analysis. Pers Psychol.
2006;59(
3):623-64.[CrossRef]
Clark
RE;
Yates
K;
Early
S;
Moulton
K. An analysis of the failure of electronic media and discovery-based learning: evidence for the performance benefits of guided training methods. : Silber
KH;
Foshay
R, . Handbook of training and improving workplace performance. Vol I: Instructional design and training delivery. Washington, DC: International Society for Performance Improvement; 2010.
Lacey
S;
Peters
A;
Sathian
K. Cross-modal object recognition is viewpoint-independent. PLoS One.
2007 Sep 12;2(
9):.
Lacey
S;
Campbell
C;
Sathian
K. Vision and touch: multiple or multisensory representations of objects?Perception.
2007;36(
10):1513-21.[CrossRef][PubMed]
Merrill
MD. A task-centered instructional strategy. J Res Tech Educ.
2007;40(
1):5-22.
Kohls-Gatzoulis
JA;
Regehr
G;
Hutchison
C. Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can J Surg.
2004 Aug;47(
4):277-83.
Yule
S;
Flin
R;
Paterson-Brown
S;
Maran
N. Non-technical skills for surgeons in the operating room: a review of the literature. Surgery.
2006 Feb;139(
2):140-9.[CrossRef]