Modern surgical practice is subject to increasing peer and public scrutiny. High-profile cases of adverse clinical events have led to improvements in systems for risk management, incident reporting, and monitoring of clinical practice and of patient outcomes1. In some countries, consultant surgeons are now required to obtain recertification to demonstrate their continuing capability to practice2,3. Some studies have demonstrated that surgeons who perform high volumes of a particular procedure achieve better outcomes, and some insurance companies have begun restricting their list of surgeons on the basis of volume of practice. This has raised awareness about the need to maintain appropriate skills, especially for rarely performed procedures4-6.
Trainee surgeons in Europe face reductions in training time and the implementation of working-hour restrictions. This is leading to the development of methods of surgical skills training outside of the operating theater. The role of simulation in medical education has become well established in general surgery and is being increasingly recognized across other specialties7-14.
In orthopaedics, arthroscopy is an irreplaceable diagnostic and interventional tool. Its breadth of use is increasing, and many advanced procedures have now been developed. Arthroscopic laboratory simulators have been used in training courses worldwide, and some studies have recently demonstrated validity, objective improvement in performance with training, and transferability of these learned skills to the operating theater15,16.
The aim of this study was to investigate the ability of experienced orthopaedic surgeons to acquire the appropriate skills to perform an unfamiliar arthroscopic procedure learned in a simulated environment and to assess their capacity to retain these learned skills.
Subjects
Six consultant orthopaedic surgeons with a subspecialty interest in lower-limb surgery were recruited from a single university teaching hospital. To be included, an individual had to currently be a consultant orthopaedic surgeon in a United Kingdom hospital, have had fellowship training in lower-limb surgery, be familiar with and regularly perform basic lower-limb arthroscopic procedures, and have had standard training in orthopaedic surgery including exposure to shoulder arthroscopy as a trainee. Exclusion criteria were any fellowship training involving shoulder surgery, a consultant practice involving shoulder surgery, any previous operative experience with arthroscopic Bankart repair, and regular performance of meniscal repair.
Simulator Training
An Alex Shoulder Professor benchtop simulator (Sawbones Europe, Malmö, Sweden) was set up in a designated skills laboratory (Fig. 1). A standard 30° arthroscope, arthroscopic camera, and display system (Smith and Nephew Endoscopy, Huntingdon, United Kingdom) were used in all cases. Immediately prior to his first session with the simulator, each surgeon viewed a standardized instructional videotape produced specifically for the study without pauses or rewinding. This videotape demonstrated the technique of performing an arthroscopic Bankart repair suture with use of the simulator. The surgeons were then allowed up to five minutes to familiarize themselves with and practice arthroscopic knot tying on a benchtop suturing model, but they were not allowed any practice sessions on the simulator. They received no additional training during this period.
Each surgeon then performed three separate simulated arthroscopic Bankart sutures on four occasions one to two weeks apart (twelve procedures in total). The number of procedures that the surgeons would perform was determined on the basis of pilot data suggesting that the initial learning curve began to plateau at twelve procedures. The surgeons received no prompting or guidance while they carried out the procedure. The adequacy of each suture was graded as "pass" or "fail" by one of the supervising authors (S.A. or G.C.H.), who used a fixed-diameter arthroscopic hook to ensure that there was no gapping between the labrum and the glenoid and that the knot was tight.
After a period of six months, the surgeons then returned to the laboratory and repeated the process, again performing the procedure twelve times spread over four sessions. During this period, they continued with their routine lower-limb practice and did not infringe on any of the exclusion criteria. They were not shown the instructional videotape again or given any instruction or reminders about how to perform the task. They were not allowed to practice on the simulator, but again they were permitted to practice arthroscopic knots on a suturing model.
Motion Analysis
During the simulator sessions, a three-dimensional electromagnetic motion tracking system (PATRIOT; Polhemus, Colchester, Vermont) was used to assess surgical performance objectively. This tracking technology has been used previously, and its feasibility, reliability, and validity have been extensively assessed during laparoscopic and open general surgical procedures9-11. We recently showed it to also be valid as a means of objective assessment of arthroscopic psychomotor skills15. The system consists of two small sensors placed in fixed positions on the dorsum of the subject's hands and an emitter that is fixed to the simulator. The output consists of the three-dimensional position (x, y, and z coordinates) of each sensor relative to the emitter as a function of time. A standardized simulator environment was maintained at all times. It was kept free from all metal objects other than the arthroscope and arthroscopic instruments. The simulator environment was checked with calibration objects while those metal objects were in situ and was found to provide a reliable output. This output was recorded by a personal computer and processed by custom software (MATLAB, version 6.5; The MathWorks, Natick, Massachusetts) to produce three outputs: the total path length of the surgeon's hands, total number of hand movements, and time taken to perform the sutures. Surgeons with greater technical ability and experience perform tasks with greater efficiency and economy of movements. Validation studies have shown this assessment tool and its outcome parameters to be capable of sensitive differentiation between surgeons of differing abilities, and those with greater technical skill perform procedures in less time, requiring fewer hand movements and a shorter total path length. These outputs have been used successfully by others10,11.
Statistical Analysis
Statistical guidance was provided by the Medical Statistics Department of our university. The primary outcome measure was the difference in the performance on the simulator between the initial study and the repeat study after the six-month interval. The data were not normally distributed, and therefore nonparametric tests were used.
The Mann-Whitney U test was used to compare the motion analysis parameters (total path length, number of hand movements, and time taken) during the first three attempts in each study with those parameters during the last three attempts in each study in order to identify any differences in performance on the simulator between these initial and final attempts. The Mann-Whitney U test was also used to compare the motion analysis parameters between the initial and repeat studies in order to demonstrate any improvement in performance, and hence learning achieved, with repetition of the task—i.e., to demonstrate a learning curve.
Source of Funding
There was no external funding for this study.
This study objectively demonstrated an initial learning curve for an unfamiliar arthroscopic technical skill performed in a simulated environment by a group of experienced surgeons. It also showed that, after six months without practice, the previous level of learned skill and performance was lost.
The concept of the learning curve for a surgical procedure has been well described17 but often only anecdotally discussed. Learning curves for performance of simulated and live surgical procedures have been demonstrated with use of various forms of assessment16,18-21.
A learning curve for arthroscopic rotator cuff repair in vivo has been acknowledged but is difficult to quantify22. To our knowledge, this is the first study to objectively demonstrate a learning curve for an arthroscopic shoulder procedure in a simulated environment.
The role of simulation in medical education is being increasingly well established. There have been a large number of studies across surgical specialties supporting the role of both laboratory and virtual reality simulators for a wide variety of open and minimal access procedures8-14,23-26. Importantly, some recent studies have demonstrated transfer validity of simulator-acquired skills to improved performance in the operating theater, answering early criticisms regarding the application of simulation to the clinical situation16,27,28.
Most simulator studies to date have largely focused on training in and assessment of simple technical skills appropriate for more junior trainees. Simulation has been particularly advocated for use at the initial steep end of a trainee's learning curve for a new procedure prior to operating on patients9. There is the potential for it to perform a similar role for established surgeons learning newly developed procedures or wishing to maintain their level of performance of more complex, infrequently performed procedures. We have shown that a simple laboratory simulator combined with an objective assessment tool can be appropriate for developing initial skills at the steep end of the learning curve for a more technical arthroscopic procedure.
Previous studies have shown that surgical outcomes are improved by surgeons performing a particular procedure with high frequency4-6. They have also shown varying degrees of retention of simulator-acquired laparoscopic surgical skills, with the variability due in part to differences in study design. All, however, have provided a snapshot assessment of performance after a time delay, rather than a repetition of the entire learning curve29-31. We found no previous studies in which the retention of arthroscopic skills was evaluated with a method similar to ours.
The demonstration in this study of the loss of the level of performance that had been achieved after the initial learning curve for a particular simulated surgical procedure confirms that newly acquired technical skills can be lost, even by experienced surgeons, without continued practice. This study was not conducted to define any level of competence, and it is worth mentioning that all of the surgeons performed adequate Bankart repairs even though they were learning the procedure. However, the study suggests that these newly acquired skills need to be practiced to maintain the best performance. This finding can be related to the well-established Fitts and Posner three-stage theory of motor skill acquisition, applied to surgical skills by Reznick and MacRae8. The theory describes an initial cognitive stage whereby the surgeon learns to understand the task but performs erratically, a subsequent integrative stage in which knowledge is translated into appropriate behavior, and finally an autonomous stage in which the performance is smooth and the surgeon no longer needs to concentrate on specific aspects of the new skill. Perhaps, when new skills are learned, an integrative stage of motor learning is achieved after completion of an initial learning curve such as the one demonstrated in our study, but if these learned skills are not firmly consolidated by continued practice, and the surgeon does not reach the autonomous stage of learning, then the level of performance is not retained.
Although this study focused on a single simulated arthroscopic shoulder procedure, that procedure was chosen to provide a generic, unfamiliar arthroscopic skill to be learned by an experienced cohort of study subjects. Thus, it seems reasonable that a similar skill-loss effect would be seen for other, similar technical skills. If that is the case, these findings raise the important point that there is a need to ensure continued repetition and practice of acquired surgical skills in order to retain them. Although we have previously shown that skills acquired in a simulated environment can be transferable to the operating theater15,16, we cannot conclude from the present study that skills acquired in an operating theater environment would also be lost. However, it is reasonable to suggest that they might. Although they will not be easy to perform, additional studies are required to objectively assess intraoperative performance of trainees and their skill retention over time in the operating theater environment.
A limitation of this study is the small sample size; however, several of the comparisons revealed highly significant values. Furthermore, the surgeons in the study were well matched in terms of general surgical experience and also specific experience with shoulder surgery and arthroscopy. While there is the potential for a type-II error when study periods are compared, the graphical data for each study period demonstrated almost identical learning curves. Further work is focusing on a larger number of surgeons and an evaluation of the frequency of practice required for skill retention.
In conclusion, this study objectively demonstrated an initial learning curve for a simulated arthroscopic Bankart suture procedure in a group of experienced surgeons and showed that, after six months without practice, the previous level of learned skill and performance was lost. The development of appropriate validated simulator exercises may provide a useful tool with which surgical trainees and established surgeons can consolidate and refresh their previously learned skills prior to carrying out procedures in the operating theater. 