Measures
Safety climate was measured by a modified version of one of the most widely used measures of safety climate, the Patient Safety Climate in Healthcare Organizations (PSCHO) questionnaire. The PSCHO is a validated questionnaire instrument5 that was developed by the Patient Safety Culture Institute at Stanford University in cooperation with the VA Palo Alto Health Care System2. This forty-two-question scale was modified by George Bilke, MD, at Dartmouth-Hitchcock Medical Center, into a shorter format and was shared with one of the authors (J.H.H.) in a personal communication6. This modified version of the PSCHO questionnaire is given in Table I.
The modified Safety Climate Questionnaire has twenty-four safety-focused questions. Responses were recorded on a 5-point Likert scale and ranged from strongly disagree to strongly agree. Problematic responses, responses implying a lack of safety climate, which included both strongly disagree and disagree, were assigned a value of —1. A neutral response was assigned a value of 0. A response of agree or strongly agree represented the perception of a positive safety climate and was assigned the value of 1. Question 2 was worded in a negative fashion, and it was reverse-scored so that all positive responses were given positive numerical representation. Given the total number of questions contained in the modified instrument, the range of possible scores was from —24 to 24. Of the 4056 possible responses in our surveys, sixteen values (0.4%) were missing. Missing responses were assigned the value of 0 to not influence the scores toward a specific conclusion. No survey contained more than two missing values.
To identify specific factors that influence residents’ safety climate, scores were subdivided by safety topic—organizational factors, communication up and down the chain of command, management's ability to respond to safety issues, safety reporting, and concerns of safety at the point of care. The organizational subscale included factors such as organizational resources, the presence of experienced personnel, peer pressure for safety, clear performance standards, monitoring of performance and compliance, the appropriateness of the level of the individuals making decisions, ongoing training, the facility's reputation for high-quality performance, and identification and management of the risks to patient safety. The organizational subscale score had a range from —12 to 12. The communication subscale involved clear lines of communication up and down the chain of command. The communication subscale score had a range of —2 to 2. The management subscale included factors that address management's capacity to restrict clinicians who are under stress, to react to unexpected changes, to promote a safety climate, to have safety backups for human errors during high-risk activities, to set an example by complying with safety guidelines, to convey clear safety goals, to conduct regular reviews of safety practices, and to understand the risks that routinely affect patient safety in their institution. Management subscale scores ranged from —8 to 8. The reporting subscale involved one item that reflected an individual's willingness to report unsafe behavior. The reporting scale ranged from —1 to 1. Last, the point-of-care subscale reflects the staff's genuine concern for patient safety and values ranged from —1 to 1.
A percentage of problematic responses of =10% was considered to represent a noteworthy deficit in safety climate, which may increase the likelihood of error and patient harm. This value was determined by author consensus.
Subjects
All postgraduate year (PGY)-2 through PGY-5 orthopaedic surgery residents of a large orthopaedic surgery resident program in a northeastern metropolitan area were invited to participate in this institutional review board-approved study. After completing the consent form, residents completed a questionnaire regarding safety at four distinct time points in 2004, 2005, 2008, and 2009. Residents had the option to include their name and postgraduate year on the questionnaire, but this was not required.
One hundred and sixty-nine (91%) of 185 residents completed the survey. There were a total of forty-three PGY-2 residents, thirty-seven PGY-3 residents, thirty-nine PGY-4 residents, and thirty-seven PGY-5 residents. Thirteen residents elected not to report their postgraduate year. Thirty-nine (89%) of forty-four residents completed the survey in 2004; forty-two (95%) of forty-four residents, in 2005; forty (83%) of forty-eight residents, in 2008; and forty-eight (98%) of forty-nine orthopaedic residents, in 2009.
Statistics
Analysis of the variance (ANOVA) was carried out with use of resident year, e.g., PGY-2, PGY-3, PGY-4, and PGY-5, to determine if the evolution of patient care responsibilities changed the perception of safety climate during residency. ANOVA was used to determine if calendar year impacted safety climate over time. A two-tailed p value of <0.05 was considered significant. Analysis of the data was performed with use of the SPSS software package (version 18.0, PASW; IBM SPSS, Chicago, Illinois).
The residents’ percentage of problematic responses, which implied a lack of safety climate, was compared with published results of other hospital personnel and naval aviators.
Sources of Funding
There was no external funding source.
One hundred and sixty-nine resident responses were recorded. Overall scores across the postgraduate year of the residents and the calendar year ranged from —13 to 24 points. The mean score and standard deviation were 12.8 ± 7.8 points. Twenty-five percent of the residents had a score of =8 points. Half of the residents had a score of =14 points, and 75% had a score of =19 points. Ten (6%) of 169 resident surveys demonstrated an overall lack of safety climate in their workplaces. These scores ranged from —3 to —13 points. Given that a neutral response may also imply a lack of a positive safety climate2, when neutral scores were included, eleven (7%) of 169 residents showed a lack of a positive safety climate.
The overall rate of problematic responses was 12.6% for residents. Table II shows the percentage of problematic responses, neutral responses, and positive safety climate responses by question. When neutral responses were included as a marker of a lack of positive safety climate2, the rate increased to 21.4%. In a study of naval aviators who completed the Naval Aviation Command Safety Assessment Survey—an equivalent safety climate survey for the military from which the PSCHO was derived—only 5.6% of pilots reported the overall absence of a safety climate7. Problematic responses for other health-care workers have been reported at 17.1% to 20.2% in other hospital populations2,7-10.
When analyzed by postgraduate year, no significant differences were detected in overall safety climate or its subscales (p > 0.40 for all).
When analyzed according to calendar year, significant differences emerged (Table III). There were significant differences between organizational subscales, communication, management, reporting, and the overall total safety climate score. The organizational subscale significantly decreased from a mean of 8.5 points in 2004 to 5.1 points in 2008 and was 7.02 points in 2009 (F = 8.1, p < 0.001). The communication subscale had decreased from 1.4 points in 2004 to 0.78 points in 2008 and was 1.2 points in 2009 (F = 3.5, p < 0.017). Management decreased from 4.7 points in 2004 to 1.0 point in 2008 and was 2.6 points in 2009 (F = 11.3, p < 0.001). Reporting of unsafe behaviors decreased from 0.8 point in 2004, to 0.6 point in 2005, to 0.1 point in 2008, and was 0.4 point in 2009 (F = 8.0, p < 0.001). Overall safety climate decreased from 16.5 points in 2004, to 15.3 points in 2005, to 7.7 points in 2008, and was 12.0 points in 2009 (F = 12.1, p < 0.001). Residents’ belief that the staff was genuinely concerned with patient safety (point-of-care subscale) decreased from a mean of 0.97 point in 2004 to 0.85 point in 2009, although this was not significant (F = 2.1, p = 0.11).
With all residents considered together, the management subscale had the lowest mean score relative to its maximum score (3.0 compared with 8.0 points). The management subscale scores were driven down by responses to two individual questions, both of which had overall negative mean response values indicating a lack of safety climate. Residents thought that management did not restrict physicians under high stress (mean, —0.30 point), and residents thought that management did not react well to unexpected changes (mean, —0.16 point).
The percentage of problematic responses for each question is reported in Figure 1. On the whole, residents reported =10% problematic responses, generating areas of concern for communication up the chain of command, regular reviews of performance, an unwillingness to report unsafe behavior, people at the wrong level making critical decisions, lack of safety backups to catch human errors, failure of senior management to understand the risks in the delivery of patient care, loss of experienced personnel, lack of good communication down the chain of command, lack of peer pressure to discourage unsafe acts, and failure to uniformly prescribe performance and safety standards. Residents reported =20% problematic responses for the failure of management to reach out to and restrict physicians under stress as well as management's ability to react to unexpected change. As reflected in the management subscale scores above, these two items were considered serious safety hazards and threats to establishing and maintaining a positive safety climate. Management's failure to restrict physicians under high stress and react well to unexpected change corresponded to an overall percentage of problematic responses of 42.0% and 39.1%, respectively.
The percentage of problematic responses for each question is shown.
The top ten most problematic responses are reported in Figure 2. This identified problematic responses of >10% in the areas of management, organization, communication, and reporting. High levels of unsafe responses were reported in nearly every subscale, with the exception of the point-of-care subscale, despite 98.8% of the residents being genuinely concerned for patient safety during care delivery.
The percentage of problematic responses for the ten most problematic areas is shown.