Database
Data were extracted from selected years of the Nationwide Inpatient Sample (NIS), a component of the Healthcare Cost and Utilization Project, which is sponsored by the Agency for Healthcare Research and Quality. Each year, the NIS contains discharge data from approximately seven to eight million hospital stays at about 1000 hospitals, randomly selected to approximate a 20% stratified sample of hospitals across the United States. The NIS includes the largest nationwide database on all-payer hospital inpatient care in the United States and contains data for patients with Medicare, Medicaid, and private insurance, as well as patients without insurance. Data are reflective of discharge diagnoses, and coding for the database is in accordance with the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM). The quality control and reliability of the NIS have been described previously7, and the accuracy is at least as good as the National Hospital Discharge Survey8. The NIS discharge data from 1998 to the present are weighted to better reflect the cross-sectional population of hospitals; thus, the results of this study are generalizable to the entire patient population undergoing hip fracture surgery in the United States.
Sample Selection
We selected the records of patients over the age of fifty-five years with an ICD-9-CM primary procedure code indicating a primary fracture of the neck of the femur (820); a closed fracture of an unspecified, intracapsular section of the neck of the femur (820.00); a closed fracture of the base of the neck of the femur (820.03); an open fracture of the base of the neck of the femur (820.13); an open fracture of the intertrochanteric section of the femur (820.31); a closed fracture of an unspecified part of the neck of the femur (820.8); or an open fracture of an unspecified part of the neck of the femur (820.9). Patients were only included in the analysis if they received any of the following surgical procedures: internal fixation of the femur without fracture reduction (78.55), closed reduction of the fracture with internal fixation of the femur (79.15), open reduction of the fracture without internal fixation of the femur (79.25), open reduction of the fracture with internal fixation of the femur (79.35), total hip replacement (81.51), or partial hip replacement (81.52). We excluded patients with ICD-9-CM diagnosis codes associated with pathologic fractures or metastatic cancer.
Predictive Variables
We outlined two primary predictor variables for the study. First, we captured data exclusively from the NIS database for the years 2001 to 2002 and grouped these findings, and we captured data from 2004 to 2005, which were also distinctly grouped. These two groups represented the data from before reform (the years 2001 to 2002) and those from after reform (the years 2004 to 2005). Second, the data were organized into three hospital teaching structures: (1) rural, (2) urban nonteaching, and (3) urban teaching. For the purposes of this study, rural and urban nonteaching hospitals were classified as nonteaching facilities, whereas urban teaching hospitals were classified as teaching facilities.
Main Outcome Measures
Outcomes included in this study were perioperative complications and mortality, length of hospital stay, hospital disposition (i.e., routine or nonroutine discharge), and hospital costs adjusted for inflation. Specific complications included central nervous system complications, respiratory complications, pneumonia, myocardial infarction, peripheral vascular complications, postoperative hypertension, hemorrhage, serum reactions, postoperative infection, systemic inflammatory response syndrome or septicemia, delirium, renal complications, digestive complications, decubitus ulcers, and transfusions. Complications were identified in the NIS by ICD-9-CM diagnosis or procedure codes and were reported as a dichotomous variable when appropriate.
Covariates
Patient-specific covariates considered in this study included age, sex, race, median household income (with use of postal zip-code data), payer source, hospital bed size, hospital region, hospital ownership or control, and Deyo index for each patient. The Deyo index summarizes patient comorbidities with use of ICD-9-CM diagnosis codes and takes into account the severity of the specific diagnoses by weighting9. The Deyo index, an adaptation of the Charlson index10, contains seventeen weighted conditions as detailed in Table I. We used the Deyo score to adjust for comorbidity and reduce potential confounding of patient health status; the utility and limitations of comorbidity scores in administrative database research have been well described11. For this study, values were calculated on the basis of percentile distribution 0-33-100. A value of 0 reflected a lower score for comorbidities, a value of 1 reflected a higher score, and a value of >1 reflected representation in a higher percentile of comorbidity within the sample.
Statistical Analysis
Descriptive statistics, including bivariate analyses of differences before reform and after reform for teaching and nonteaching hospitals, were used within the study, and comparative differences in age, sex, race, median household income (with use of postal zip-code data), payer source, hospital bed size, hospital region, hospital ownership or control, and Deyo index were calculated. Race is routinely not collected by selected hospitals, which results in a high level of missing values. To compensate for missing values, models were run with and without imputation; no difference was found.
To determine if there were differences in the reporting of complications between teaching facilities and time periods, we performed a multivariate comparison of complications among all four classified groups: (1) nonteaching hospitals before reform, (2) nonteaching hospitals after reform, (3) teaching hospitals before reform, and (4) teaching hospitals after reform. Chi-square analyses were used to compare the data; cost and length of stay were analyzed with analysis of variance when the data were normally distributed and with the Kruskal-Wallis test when they were not. Because differences in complications between years could be associated with changes in reporting strategies or administrative error, we opted to measure the trend or change in complications from before and after reform for nonteaching hospitals and before and after reform for teaching hospitals.
The first step involved two logistic regression analyses: one analyzing data before and after reform for nonteaching hospitals and one analyzing data before and after reform for teaching hospitals. Both models involved adjustments for the potential confounders of age, sex, race, median household income (with use of postal zip-code data), payer source, hospital bed size, hospital region, hospital ownership or control, and Deyo index, and both used data from before reform (and the corresponding teaching or nonteaching classification) as the reference variable. From the logistic regression modeling, we captured the beta coefficients and standard error measures, which represent the estimated average change in standard deviation units, unique to each outcome variable.
To determine whether differences between teaching and nonteaching hospitals before and after reform were significant, we performed a bivariate analysis of beta or slope coefficients across groups. By analyzing beta coefficients and standard errors between teaching and nonteaching facilities, we endeavored to determine if findings among each outcome variable of the two facility types were coincidental in nature (a reflection of reporting changes over time).
Furthermore, rather than assuming equality of reporting in teaching and nonteaching facilities, which can lead to misleading or invalid results, we assessed the residual variation of each outcome variable by calculating for delta hat12. Residual variations or residuals are observable estimates of unobservable statistical error within a sample. No residual error is represented as 0% error, whereas numbers that are positive or negative, and different from zero, reflect a probability of error specific to that number. A delta hat is used to measure symmetrical error in the reporting of data. Because we used the NIS database, and since administrative databases run the risk of variability in coding3,5, we considered variations of <15% as low group residual variation and deemed values below this figure as acceptable.
Source of Funding
There was no external funding source for this study.
After data acquisition, 48,430 patients were captured for analysis; 31,002 were from nonteaching facilities and 17,428, from teaching facilities. A number of variables demonstrated significant differences in simple demographic data points (e.g., age, sex, and payer source) from before reform to after reform. The Deyo scores that reflect the severity of patient comorbidity were higher after reform for all patients, suggesting that the patient group as a whole had greater comorbidity than the comparative group before reform. A table in the Appendix outlines the key univariate clinical statistics of the patient sample.
A table in the Appendix provides the multivariate comparisons of complications among the four classified stratifications. Among the four groups, a number of outcome variables were significantly different, including respiratory, cardiac, other, and renal complications; pneumonia; transfusions; systemic inflammatory response syndrome; delirium; decubitus ulcers; routine discharge; length of stay; and inflation-adjusted costs. In most cases, greater complications or longer length of stays were seen after reform.
Two separate regression analyses calculated variations in perioperative morbidity and mortality between teaching and nonteaching facilities in the two time periods before and after reform. For both analyses, the beta coefficients and standard errors were input for bivariate analyses. Bivariate analyses identified significant differences over time between teaching and nonteaching facilities with regard to pneumonia, presence of hematoma, transfusion, renal complications, routine discharge, length of stay, and inflation-adjusted costs (Table II). In all cases, there was an increase in incidence in teaching facilities. In all situations, teaching facilities after reform were associated with greater frequencies of complications, costs, or nonroutine discharges compared with teaching facilities before reform. As a graphic example of one of the selected outcomes, Figure 1 shows the relation between the incidence of pneumonia in teaching facilities compared with nonteaching facilities both before and after reform, demonstrating the significantly greater relative increase in incidence found after reform in teaching facilities.
The delta-hat assessment of residuals (see Appendix) supports the finding that the differences observed between teaching and nonteaching hospitals are not associated with unobserved heterogeneity (residual variation) between the comparative groups. In all cases of variables that were significant, residual variation between the two groups was <15%. We were unable to compare residuals for length of stay and inflation-adjusted costs because of the data type.
Implementation of resident duty-hour reform by the ACGME in 2003 was met with some concern, protest, and skepticism13,14. The intention of regulation was to counteract the potential negative effects of sleep deprivation and fatigue on patient care and learning15. There is little empirical evidence to support the assertion that this reform has improved patient care3,16. This is particularly true in orthopaedic surgery, where there are no data on which to draw conclusions. A key finding of this study was that the implementation of reform was associated with an accelerated rate of increase in morbidity following hip fracture surgery at teaching institutions. Thus, we could not support our null hypothesis.
The internal medicine literature contains several studies that support the contention that the ACGME restrictions were not deleterious to the survival of medical patients. In parallel studies, one group found no evidence of increased mortality in Medicare patients and patients in U.S. Veterans Affairs hospitals2,3. Subgroup analysis actually suggested a relative improvement in mortality for several medical conditions, although no changes were seen specifically in surgical patients2. These studies were limited to mortality rates and did not attempt to examine morbidity associated with treatment.
Another recent study, with use of the same NIS database and similar statistical methods as were used in our analysis, demonstrated some evidence of mortality reductions for medical patients in teaching hospitals following duty-hour restrictions5. However, that study did not demonstrate a reduction in mortality for surgical patients and, in fact, demonstrated a trend toward increasing mortality. Although it was not significant, the authors found a relative increase in mortality of 3.77% for surgical patients associated with duty-hour restrictions. The authors noted that a smaller sample size for surgical patients may have limited their power to detect significant differences. Markers of morbidity were not examined in this study.
Data suggesting adverse consequences in patient care associated with resident duty-hour reform are beginning to emerge. Consistent with our results, some recent studies have suggested that limiting work hours has had an adverse impact on patient outcome. One recent study found suboptimal medication administration and longer patient length of stay when patients with heart failure in Veterans Affairs hospitals were admitted by "short call" residents compared with the patients admitted by residents taking more traditional "long call."17 Other observational studies have also demonstrated negative effects of discontinuity of care on the quality of patient care and length of stay18-20, although conflicting analyses have suggested a neutral or positive impact associated with restricting work hours21-23.
Orthopaedic surgery may present several unique challenges that make extrapolation from the internal medicine literature untenable. As noted by other authors, a reduction in duty hours may have the greatest impact in the subspecialties in which residents worked the most hours prior to reform (e.g., surgery)2,3. If the number of resident providers remains fixed and they are each permitted to work fewer hours, then the number of available resident surgeons at any given time must necessarily decline, a problem that is amplified in specialties with relatively few resident providers. Any potential benefits in terms of reduced fatigue may be offset by increased work intensity3.
The changes in medical coverage created by resident duty-hour reform impact two specific aspects of care that have had little investigation. The first aspect is the availability of senior-level medical providers for supervision of on-site junior resident medical coverage. Many programs have implemented a night-float system of call for on-site coverage. While this addresses the duty-hour issue of junior-level residents, it does not address the issue of senior-level resident coverage. We found no report specifically addressing the changes in senior-level supervision of the on-site residents following resident duty-hour reform. Furthermore, given the reduction in the number of residents physically in the hospital at any given time, the volume and responsibility of call coverage per resident have likely increased.
The second aspect of medical coverage is an increase in the number of handoffs by medical providers to maintain continuity of care. It is our anecdotal experience that continuity of care has become more challenging in the orthopaedic teaching environment following duty-hour limitations. Handoffs, particularly problematic in patient care and known to increase the risk of adverse events14,17,18,22,24-26, appear to occur relatively more frequently in the surgical services after reform. Several handoffs and cross-coverage in care potentially occur during the treatment of a patient with a hip fracture and may involve members of the house staff team in various surgical specialties and stages of training. Often, the resident gaining information during the emergency department evaluation differs from the resident assisting in the operative procedure; the night-float resident then covers the patient in the immediate postoperative period, only to again transfer care to the inpatient orthopaedic service the following morning. Orthopaedic patients, particularly those admitted through the emergency department, may be subject to more than the two daily handoffs for medical patients that have been reported in the literature26. Furthermore, handoffs in surgical disciplines may be different from those in other areas of medicine, as acute clinical conditions such as hematoma or compartment syndrome that may evolve rapidly are best handed off at the patient's bedside, where both the resident who is coming on duty and the resident who is going off duty can examine the patient.
We used nonteaching hospitals as a control when assessing changes in morbidity before and after resident duty-hour reform. By using nonteaching hospitals as a control, we not only examined changes at teaching facilities over time but also adjusted for variations associated with health-care changes, administrative database anomalies, or other elements not associated with actual patient-care trends. We then compared the slope coefficients across groups. Unlike linear regression coefficients, however, coefficients in these binary regression models are confounded with residual variation (unobserved heterogeneity). Differences in the degree of residual variation across groups can produce apparent differences in coefficients that are not indicative of true differences12. To control for the possible differences in residual variation between groups, we calculated the variable delta hat as described by Allison12. Our assessment of residuals for each outcome measure ensures that our findings are associated with true change and are not confounded by unobserved heterogeneity within the two groups.
Although the statistical methods used in this study attempted to control for confounding and to adjust for variations in groups, group differences can still influence outcomes. We acknowledge that a small, disproportionate increase in patients with a Deyo score of >1 was observed across the study time interval in the teaching hospitals (a 3.43% increase in nonteaching hospitals compared with a 4.86% increase in teaching hospitals). Furthermore, despite our attempts to control for this possibility statistically, earlier referral of sick patients to teaching hospitals might independently contribute to the observed escalated rate of complications in teaching hospitals.
Some additional limitations to this study require comment. Shortcomings of the NIS database have been recognized3,5. The NIS database includes in-hospital complications only, and any adverse perioperative outcomes that may have occurred after discharge would not be reflected in our analysis. As with all administrative databases, there is the inherent risk of error based on incorrect or absent coding of data. One cannot be absolutely certain that the patient underwent the coded procedure or had complications from it. Furthermore, the NIS classifies a hospital as "teaching" if the facility has any type of residency program, not just orthopaedic surgery. Thus, some of the patients classified as having received their care in the setting of a teaching hospital may have had no interaction with an orthopaedic surgery resident during their treatment, and it is possible that the NIS variable led to misclassification and incorrect assignment of teaching status. Finally, we only analyzed a limited time period soon after duty-hour restrictions were implemented, and the higher incidence of complications may reflect experimentation by the teaching programs with various strategies to accommodate to the changes. This association may not hold up over time as systems are implemented to effectively deal with duty-hour restrictions.
Resident performance and health-care delivery are complex processes, particularly within the intricate system of a teaching hospital6,16. Our investigation identified that the rate of change of perioperative morbidity in patients with a hip fracture increased significantly in teaching hospitals following resident duty-hour reform, and we cannot accept our null hypothesis. It should be emphasized that our findings demonstrate an association and do not imply causality. Further studies that examine patient outcomes in orthopaedic surgery following work-hour restrictions are clearly needed. Although it has been suggested that new data would be unlikely to reverse duty-hour reforms27, empirical evidence is needed to develop a comprehensive understanding of the impact of resident duty-hour reform on patient care before any further changes in duty-hour regulations are considered. The short and long-term impact of duty-hour restrictions on both patient care and resident education in orthopaedic surgery remains unclear.