Abstract
Background:The Journal of Bone and Joint Surgery, American
Volume (The Journal) recently initiated a section called
"Evidence-Based Orthopaedics." Furthermore, a level-of-evidence
rating is now used in The Journal to help readers in clinical
decision-making. Little is known about whether this recent emphasis has
influenced surgeons' perceptions about and competence in evidence-based
medicine. Therefore, we examined perceptions and competence in evidence-based
medicine among Dutch orthopaedic surgeons.
Methods: Members of the Dutch Orthopaedic Association were surveyed
to examine their attitudes toward evidence-based medicine and their competence
in evidence-based medicine. We evaluated competences using a newly developed
instrument tailored to surgical practice.
Results: Of the 611 members, 367 surgeons (60%) responded.
Orthopaedic surgeons welcomed evidence-based medicine. Practical
evidence-based medicine resources were perceived as the best method to move
from opinion-based or experience-based to evidence-based practice. Four
variables were significantly and positively associated with the competence
instrument: (1) a younger age, particularly between thirty-six and forty-five
years (p = 0.007), (2) experience of less than ten years (p = 0.032), (3)
having a PhD degree (p < 0.001), and (4) working in an academic or teaching
setting (p = 0.004). The majority of the respondents were aware of The
Journal's evidence-based medicine section (84%) and level-of-evidence
ratings (65%), and 20% used The Journal's evidence-based medicine
abstracts in clinical decision-making. This increased awareness of
evidence-based medicine was also reflected in the frequent use of Cochrane
reviews in clinical decision-making (27% of the respondents). Surgeons who
used and those who were aware of but did not use The Journal's
evidence-based medicine abstracts or Cochrane reviews in clinical
decision-making had significantly higher competence instrument scores than
those who were unaware of these resources (p = 0.03 and p < 0.001,
respectively).
Conclusions: Evidence-based medicine is welcomed by Dutch
orthopaedic surgeons. The recent emphasis on evidence-based medicine is
reflected in an increased awareness about The Journal's
evidence-based medicine section, levels of evidence, and the largest
evidence-based medicine resource: the Cochrane reviews. Younger orthopaedic
surgeons had better knowledge about evidence-based medicine. The development
and use of evidence-based resources as well as preappraised summaries such as
The Journal's evidence-based medicine abstracts and Cochrane reviews
were perceived as the best way to move from opinion-based to evidence-based
orthopaedic practice.
To give patients the optimal treatment and to control health-care expenses,
orthopaedic surgeons must consider a shift in paradigm from traditional
"eminence-based" and experience-based practice to evidence-based
practice—one which involves question formulation, validity assessment of
available studies, and appropriate application of the best available evidence
to individual
patients1.
The need for evidence-based medicine has been actively promoted by the
Dutch Orthopaedic Association (Nederlandse Orthopaedische Vereniging) and the
Dutch Trauma Association (Nederlandse Vereniging voor Orthopaedische
Traumatologie)2.
The Journal of Bone and Joint Surgery, American Volume (The
Journal) published a series entitled "User's Guide to the
Orthopaedic Literature" to teach critical appraisal to orthopaedic
surgeons3, and a new
section called "Evidence-Based Orthopaedics" was
introduced4. This
section is specifically devoted to busy clinicians with real-world questions;
the hope is that these abstracts will help to improve the quality of
orthopaedic
practice4.
Furthermore, a level-of-evidence rating is now used in The Journal to
help readers in clinical
decision-making5.
English is the medical language in the Netherlands, and members of the Dutch
Orthopaedic Association have access without charge to full-text articles at
by means of a link at the Association's web site:
6.
Critics of evidence-based medicine often report its inability to individualize
patient care and the interference with physician autonomy that can result from
our dependence on
it7.
The practice of evidence-based medicine involves the application of a
specific nomenclature and skills toward the integration of the current best
research evidence, an understanding of the clinical circumstances (resources),
and patients' values and preferences. At present, very little is known about
the attitudes of orthopaedic surgeons toward evidence-based medicine, the
extent of their skills to access and interpret evidence, the barriers to
moving from opinion-based to evidence-based practice, and the additional
support necessary to incorporate evidence-based medicine into everyday
practice.
Given the need to increase the application and understanding of
evidence-based medicine principles in orthopaedics, the objective of this
study was to determine the attitude of orthopaedic surgeon members of the
Dutch Orthopaedic Association toward evidence-based medicine. We also aimed to
test self-rated skills in evidence-based medicine compared with a newly
developed instrument for the assessment of competence in evidence-based
medicine8. It was
hypothesized that surgeons lacked awareness of evidence-based medicine
resources and basic understanding of critical appraisal issues, but welcomed
evidence-based medicine training. We developed a priori hypotheses regarding
potentially important predictors of surgeons' understanding of evidence-based
medicine. These included, in order of importance, the age of the surgeons,
academic practice, teaching hospital practice, involvement in research,
postgraduate training (a PhD degree), and the number of surgeons in a practice
group. We hypothesized that surgeons who used the evidence-based medicine
section in The Journal, or the levels of evidence in The
Journal, and the Cochrane database to help in clinical decision-making
would have more knowledge about evidence-based medicine as reflected in a
higher score on the competence instrument than would surgeons who did not use
these evidence-based medicine resources. We also hypothesized that surgeons
would have a high score for self-rated knowledge and a moderate score on the
competence instrument as a reflection of socially desirable answers or an
over-rating of their own knowledge.
Questionnaire Selection and Development
Item Generation
We searched MEDLINE for articles reporting results of questionnaires on
attitudes of doctors toward evidence-based medicine. We used the questionnaire
developed by McColl et al. as a foundation for our questionnaire since it was
cited most frequently and was used in different specialties (cited 161 times
in the Web of Science in January
2006)9. The survey
developed by Hanson et al. was the only one found to determine the need for
education in evidence-based medicine among orthopaedic
surgeons10.
Orthopaedic surgeons in Switzerland, the Netherlands, and Canada and an
epidemiologist participated in the development of the questionnaire. The
survey that was used combined and modified the questionnaires of McColl et al.
and Hanson et al. for administration to Dutch orthopaedic surgeons, with the
permission of those
authors9,10.
The questionnaire consisted of visual analogue scales to determine the
attitudes of the surgeons toward evidence-based medicine. The scores on the
visual analogue scale ranged from a minimum of 0 to a maximum of 100. Closed
questions were used to assess the surgeons with regard to their awareness of,
and perception of the usefulness of, extracting journals, review publications,
and databases relevant to evidence-based medicine; their ability to access
MEDLINE or other bibliographic databases on the World Wide Web; their
understanding of technical methodological terms; and their views on how best
to move from eminence-based or opinion-based practice to evidence-based
practice. The questionnaire was translated into Dutch by two of the authors
(R.W.P. and J.A.M.) and by an independent native English-speaking person.
Dutch doctors are familiar with English terminology since English is the basic
medical language in the
Netherlands6,11,12.
The translated version was translated back to English and checked against the
original questionnaire to prevent translation errors. The English version of
the questionnaire can be found in the Appendix.
Pretesting and Validity Assessments
We pretested the questionnaire with an independent group of three
orthopaedic surgeons and one epidemiologist to evaluate how the questionnaire
addressed the following questions. First, does the questionnaire as a whole
appear to adequately address the attitudes of orthopaedic surgeons toward
evidence-based medicine (i.e., does it have face validity)? Second, do the
individual questions adequately reflect the following four domains: (1)
attitudes toward evidence-based medicine, (2) views on the perceived major
barriers to practicing evidence-based medicine, (3) views on how best to move
from opinion-based to evidence-based medicine, and (4) competence in
evidence-based medicine terminology? The surgeons also commented on the
clarity and comprehensiveness of the questionnaire. After adjustments were
made, the questionnaire was tested again by a different group of three
independent surgeons.
Demographic Data
The respondents were asked to provide their gender and age and to indicate
whether they were a resident or consultant, the type of practice (academic,
teaching, private, or other type of hospital), years of experience, part-time
or full-time appointment, research activities, highest level of training or
experience in research, subspecialty, and the size of the practice (number of
colleagues and number of new patients per year).
The demographic data from a random sample of ten nonrespondents were
identified on the Dutch Orthopaedic Association web site, where details about
members can be found (only by Dutch Orthopaedic Association members), and in
the census of the Dutch Orthopaedic Association.
Questionnaire Administration
No formal approval of an ethics committee or internal review board was
required for this study. Completion and return of the questionnaire was
interpreted as surgeon consent for participation. The questionnaire was sent
to all member surgeons of the Dutch Orthopaedic Association. Each surgeon
received a package (a ten-page survey, cover letter, and prepaid return
envelope) by mail. The return envelope was coded to enable the registration of
responders. The questionnaire was separated from the return envelope and was
processed in a blinded manner. The cover letter for the questionnaire included
a definition of evidence-based medicine as the "conscientious, explicit
and judicious use of current best evidence in making decisions about the care
of individual patients. Its practice means integrating individual clinical
expertise with the best available external clinical evidence from systematic
research."9,13
Methods Used to Increase Response Rates
At the Annual Meeting of the Dutch Orthopaedic Association in January 2005,
a slide was projected after the presentations of the scientific meeting to
inform all members that they would be contacted to fill out the survey in the
coming months. In April 2005, all members received the survey by first-class
mail. A prepaid return envelope was provided. All nonresponders received a
reminder with a new survey and new stamped return envelope after four months.
The members who still did not respond were given the opportunity to fill out a
web-based electronic version in November 2005. A reminder was sent by e-mail
to all
nonresponders14.
Self-Rated Methodological Knowledge Score
We calculated a self-rated knowledge score on the basis of the respondent's
understanding of ten methodological terms adapted from Oliveri et al. and the
original questionnaire of McColl et
al.8,9.
The score was calculated for each respondent on the basis of the average of
the rank values applied to each type of possible answer: (1) "It would
not be helpful for me to understand," (2) "I don't understand, but
I would like to," (3) "Some understanding," and (4)
"Yes, I understand and I could explain to others." A higher score
for self-rated methodological knowledge (with a maximal possible score of 4
and a minimal possible score of 1) reflects a better knowledge about
methodological
terms8. This
self-rated methodological knowledge score was not used in the competence
instrument.
Score of Knowledge of Evidence-Based Medicine: Competence
Instrument
A false-positive result, also called a Type-I error, exists when a test
incorrectly reports that it has found a positive result where none really
exists. The hazard of testing multiple comparisons will yield false-positive
results15. In a
multiple question survey, many different comparisons can be made, hence
increasing the risk of false-positive
results15. To limit
false-positive results, we developed an instrument to describe surgical
evidence-based medicine competence by grouping relevant questions. The
competence instrument was built on the surgeons' knowledge of the following
methodological safeguards known to prevent bias in medical
research13:
blinding and randomization. The instrument also consisted of knowledge of the
levels of evidence in the medical
literature5. All
questions were relevant to the evidence-based practice in surgery. The minimum
possible score was 0, and the maximum possible score was 46. The score was
transformed into percentages of the maximum score. The coding scheme for the
competence instrument is provided in the Appendix.
Statistical Analysis
Categorical data were reported as frequencies, percentages, and 95%
confidence intervals. Continuous data were reported as the mean and standard
deviation. Whenever the distribution of responses for a particular item in the
questionnaire had multiple empty cells, we collapsed the related categories in
that particular item to achieve a more uniform distribution of responses. We
were interested in being able to compare surgeons' knowledge of evidence-based
medicine (the competence instrument score) using statistical methods to adjust
for potential variables related to the participating surgeons'
characteristics. A simple regression model would not permit the simultaneous
comparison of the surgeons' competence instrument scores toward evidence-based
medicine while controlling for potential confounding
factors16. One
solution was to hypothesize that data were likely correlated among
participating surgeons and among practices for each question. Generalized
estimating equations are estimation methods that can be used to estimate
parameters for correlated cross-sectional data with multiple observations
within one
unit16-19.
The participating surgeon was considered the unit, and the responses to the
questions on the knowledge of evidence-based medicine are repeated measures
within one participating surgeon. Univariable analysis and a multivariable
analysis with use of generalized estimating equations were performed to model
the competence instrument score as a function of age, gender, type of
practice, position, type of employment, years of experience, level of
training, and research involvement in the past
year20,21.
Generalized estimating equations were chosen in order to incorporate the
possible correlation among competence instrument scores within the same
practice size. The results of generalized estimating equations were expressed
as a coefficient, corresponding standard error, 95% confidence interval, and
associated p value. Variables that met the significance level of <0.1 in
univariable analysis were included in the multivariable analysis.
Goodness-of-fit was assessed with use of the likelihood ratio
statistic19,21.
Tests of associations between the competence instrument score and the
self-rated knowledge score were performed with use of a partial correlation
coefficient when the above variables were held constant. One-way analysis of
variance was used to compare the mean competence instrument score of the
respondents with regard to The Journal's evidence-based medicine
section, The Journal's level-of-evidence ratings, and the Cochrane
reviews. The Bonferroni test was used to adjust for multiple comparisons and
determine which means
differed15. All
analyses were conducted with use of SPSS (version 14.0; SPSS, Chicago,
Illinois) and Stata software (version 8.0; Stata, College Station,
Texas)22.
Characteristics of the Respondents
Of the 611 members of the Dutch Orthopaedic Association who received the
questionnaire, 367 (60%) responded; 231 (38%) responded within two months, 114
(19%) responded after the second mailing, and twenty-two (4%) filled out the
web-based survey. Of the 367 returned questionnaires, 366 had sufficient data
available for analysis. The typical respondent was male (91%), a faculty
member (69%) in a nonteaching hospital (55%), and worked full-time (94%) in
general orthopaedic surgery (71%) (see Appendix). Ninety-eight percent of the
respondents had Internet access at home as well as in their office, and 96%
had Internet access in their local medical library.
Our random sample of ten non-respondents revealed similarities to the
respondents with respect to age, male gender, working in a nonteaching
hospital, having a PhD degree, and being a consultant.
Attitudes of Orthopaedic Surgeons Toward Evidence-Based Medicine
The responding Dutch orthopaedic surgeons were positive toward the current
emphasis on evidence-based medicine (mean score [and standard deviation], 68
± 17.7; 95% confidence interval, 66.6 to 70.4). Practicing
evidence-based medicine was perceived to improve patient care (mean score, 71
± 17.8; 95% confidence interval, 69.7 to 73.4). Surgeons disagreed that
evidence-based medicine was of limited value for orthopaedic surgery (mean
score, 44 ± 24.7; 95% confidence interval, 41.2 to 46.4). Furthermore,
surgeons differed in their opinions with regard to the amount of time demands
imposed by evidence-based medicine in a busy surgical practice (mean score, 53
± 26.0; 95% confidence interval, 50.1 to 55.5)
(Fig. 1).
Views on How to Move from Opinion-Based to Evidence-Based
Practice
Of the 367 respondents, 52% used critical appraisal of the primary
literature and 43% were interested in using this evidence-based medicine
method in the future. Sixty-five percent used evidence-based summaries,
including Cochrane reviews and The Journal's evidence-based medicine
abstracts. The majority of the respondents (84%) used evidence-based practice
guidelines or protocols, and 76% were interested in using this method in the
future. Surgeons perceived the use of evidence-based practice guidelines as
most helpful for the orthopaedic practice.
Self-Rated Knowledge of Methodological Terminology of Orthopaedic
Surgeons
Although 10% of the respondents had no understanding of evidence-based
medicine terminology, many (40%) had some understanding and 39% claimed they
could explain the terms to others. This resulted in a mean self-rated
knowledge score of 3.25 ± 0.55 (95% confidence interval, 3.20 to 3.31).
Transformed into percentages, this represented 81.3% of the maximum possible
score (Table I).
Competence Instrument
The mean score on the competence instrument for all respondents was 65%
± 18% (95% confidence interval, 63.4 to 67.0). Univariable generalized
estimating equations analysis revealed that age, position, type of practice,
years of experience, and having a PhD degree were significantly associated
with the competence instrument score while adjusting for the size of practice
(p = 0.001 for all) (see Appendix). The type of employment, gender, type of
involvement, and having evidence-based medicine training were not significant
factors in relation to the competence instrument score. Multivariable analysis
of generalized estimating equations, adjusted for the size of practice,
revealed that younger age, particularly between thirty-six and forty-five
years (coefficient, 15.3; 95% confidence interval, 4.1 to 26.6; p = 0.007),
experience of less than ten years (coefficient, 3.5; 95% confidence interval,
0.3 to 6.7; p = 0.032), having a PhD degree (coefficient, 3.8; 95% confidence
interval, 2.6 to 5.1; p < 0.001), and working in an academic setting
(coefficient, 2.7; 95% confidence interval, 0.9 to 4.6; p = 0.004) were
significantly and positively associated with the competence instrument score
(Table II). With respect to the
type of position, the competence instrument score was similar between
consultants and residents, but it was significantly greater for researchers (p
= 0.009).
Self-Rated Knowledge Score Compared with Competence Instrument
Score
Although it was not strong, a significant and positive correlation
(correlation coefficient = 0.320, p < 0.001) was found between self-rated
knowledge scores and competence instrument scores while controlling for age,
gender, position, type of practice, type of employment, years of experience,
type of involvement and level of training (having a PhD degree or completing
an evidence-based medicine course), area of specialty, and practice size.
However, the transformed mean self-rated knowledge score (81.3%) showed a
16.3% overestimation of knowledge compared with the mean competence instrument
score (65%).
Competence Instrument Score Compared with Use of the
Evidence-Based Medicine Section and Levels-of-Evidence Ratings in The
Journal and the Cochrane Reviews
Respondents were more aware of The Journal's evidence-based
medicine section (84%) and Cochrane reviews (84%) than they were of The
Journal's level-of-evidence ratings (65%). Surgeons who used The
Journal's evidence-based medicine section for clinical decision-making
had significantly higher competence instrument scores than those who did not
use this educational resource (mean scores, 69.4% and 63.4%, respectively; p =
0.03). We found a trend toward increased competence instrument scores for
evidence-based medicine among the surgeons who also understood and used
The Journal's levels of evidence for clinical decision-making
compared with those who were unaware of the levels (69.3% and 61.8%,
respectively; p = 0.055). For the Cochrane reviews, the linear trend in the
increase of mean competence instrument score was evident
(Fig. 2). The mean competence
instrument score was significantly higher for those who used evidence-based
approaches in clinical decision-making (69.5%, p < 0.001) and those who
were aware of evidence-based approaches but did not use them in daily clinical
decision-making (65.6%, p = 0.001) compared with those who were not aware of
evidence-based approaches (56.2%) (see Appendix and
Fig. 2).
Principal Findings
Our study had five principal findings. (1) Four variables are positively
associated with competence in evidence-based medicine terminology: a surgeon
who was younger, had less than ten years of experience, had a PhD degree, and
who worked in an academic or teaching setting had better knowledge about key
subjects in evidence-based medicine. (2) Surgeons' self-rated knowledge about
terminology relevant to evidence-based medicine correlated with knowledge
about key issues in evidence-based medicine, but was overestimated by 16.3%.
(3) More than 50% of the respondents had misconceptions about key
methodological aspects in surgical trials. (4) The majority of the respondents
(84%) were aware of The Journal's evidence-based medicine section,
and 20% used The Journal's evidence-based medicine abstracts in
clinical decision-making. This increased awareness in evidence-based medicine
was also reflected in a frequent use of Cochrane reviews in clinical
decision-making (27% of the respondents). Surgeons who used The
Journal's evidence-based medicine abstracts and Cochrane reviews in
clinical decision-making had better knowledge about important issues relevant
to evidence-based medicine. (5) Practicing evidence-based medicine was
perceived to improve patient care in orthopaedic surgery.
Strengths and Weaknesses of the Study
Our questionnaire survey had a response rate of 60%. Response rates in
previous questionnaires describing orthopaedic surgeons' knowledge about
evidence-based medicine have ranged from 38% to
50%10,23.
We are not informed about 40% of the members of the Dutch Orthopaedic
Association who did not respond. Not responding to this survey may have been
due to skepticism toward the recent emphasis of evidence-based medicine.
Nevertheless, the demographic data of a random sample of ten nonresponders did
not differ from those of the responders.
It is known that self-rating of knowledge among general practitioners leads
to overestimation of actual
knowledge24.
Responses to our questions may reflect socially desirable answers. Therefore,
we compared the self-rated knowledge score with a newly developed competence
instrument. The questionnaire used was tested for face and content validity
prior to administration. Our study may thus reflect true knowledge about
important issues relevant to evidence-based medicine. However, this survey has
a self-report format and there are no data on whether the surgeons actually
had higher utilization of evidence-based medicine in their practices.
Furthermore, we did not formally test the respondents' understanding and
ability to explain meta-analysis or publication bias. Therefore, inferences
should be interpreted with care and warrant further research.
Our study respondents were Dutch orthopaedic surgeons. Results may differ
for other specialties or geographic locations. Dutch doctors are known for
their ability to speak and understand English since English is the basic
medical language in the
Netherlands6,11,12.
We chose the Cochrane database as a reference standard since Cochrane reviews
are of high quality and the Cochrane database is the largest database relevant
to evidence-based health
care25,26.
Furthermore, two Cochrane groups are devoted to subjects relevant to
orthopaedic
practice27,28.
Previous Literature
Attitudes Toward Emphasis of Evidence-Based Medicine and Views on How
to Move from Opinion-Based to Evidence-Based Practice
General practitioners in the United Kingdom also perceived evidence-based
guidelines as the best way to promote evidence-based medicine in the late
1990s9. Dutch
orthopaedic surgeons corresponded to British general practitioners in their
positive attitude toward evidence-based
medicine9. The
majority of the respondents in our study had similar thoughts and found that
practical evidence-based resources (from the Dutch Orthopaedic Association),
pre-appraised summaries (The Journal's evidence-based medicine
abstracts), and Cochrane reviews are the best way to move from opinion-based
medicine to evidence-based-medicine. A recent survey among urologists reported
similar widespread agreement with the notion that the practice of
evidence-based medicine improved the quality of health
care29.
Self-Rated Methodological Knowledge
The self-rating of skills in evidence-based medicine came under scrutiny
after the validation study by Young et
al.24. To our
knowledge, the use of self-rating of skills has not been compared with other
key issues in understanding evidence-based medicine. We could not find a
significantly higher score in the self-reported knowledge scoring system as a
reflection of socially desirable answers, or an overestimation of knowledge.
Nevertheless, our findings did show a 16.3% overestimation on self-reported
knowledge compared with our competence instrument. Correlation cannot adjust
for systematic overrating due to the inflation of scores. These findings
should be interpreted with caution and may indeed be a reflection of socially
desirable answers.
Competence in Evidence-Based Medicine: Competence Instrument
To our knowledge, this is the first survey describing univariable analysis
and a multivariable analysis with use of generalized estimating equations to
model an evidence-based medicine competence instrument score. The
questionnaire of McColl et al. served as a foundation for several
questionnaires evaluating attitudes toward and knowledge about evidence-based
medicine9. Other
studies have described the validation of questionnaires to test competence in
evidence-based
medicine30,31.
Those questionnaires were not tailored to items relevant to evidence-based
practice in surgery. Surgical randomized trials differ from pharmaceutical
trials, and testing of surgeons' knowledge may need a different approach as
the approach is different for assessing study quality in surgical
trials32-35.
Despite the increased awareness among orthopaedic associations and journal
editors, a recent survey of 543 participants in an AO course suggested that a
large number of survey respondents had misconceptions concerning important
aspects of evidence hierarchy and common terminology in study
design10. In our
study, a comparable number of respondents (10.3%) reported that they had never
heard about randomized controlled trials as a study design. Our respondents
were more familiar with the term blinding; only 7.2% reported that
they never had heard about blinding in patient research. However,
misconceptions about who can be blinded in a double-blinded surgical trial
were numerous. More than 50% thought that surgeons could be blinded. Although
it is obvious that surgeons cannot be blinded in a surgical trial, the
interpretation of blinding terminology in patient research has been shown to
be confusing for
physicians36.
Alternatively, the related question(s) in our questionnaire could have been
vague, or response bias could have resulted in these findings. Nonresponders
may have judged the question too obvious to answer. Results from self-reported
questionnaires should be interpreted with caution.
Other reports have revealed a substantially lower awareness of the Cochrane
database than we found in our survey. Interestingly, respondents in previous
reports utilized Cochrane reviews to help in clinical decision-making far less
frequently, ranging from 3% to 9% compared with 27.1% of our
respondents8,9,29,37,38.
Only 16.3% of the respondents in our study were unaware of the existence of
Cochrane reviews. Furthermore, users of Cochrane reviews were shown to have
higher scores on the competence instrument
(Fig. 2). We do not know
whether the use of Cochrane reviews or The Journal's evidence-based
medicine section resulted in a higher score on the competence instrument, or
whether surgeons with better knowledge about evidence-based medicine were more
likely to use these evidence-based medicine resources. The recent support of
evidence-based medicine by the Dutch Orthopaedic Association may have resulted
in this greater awareness of Cochrane reviews. Unfortunately, our data are too
sparse to make strong inferences in this direction. However, evidence-based
surgery has been given an important place in the Netherlands in recent
years39.
Interestingly, Dutch general surgeons embraced the concepts of randomized
controlled trial design as early as the
1980s40.
The literature supports the notion that older surgeons have difficulties
learning new technical
skills41. An
age-dependent decline in training-dependent plasticity was reported,
suggesting that the ability of a healthy aging motor cortex to reorganize in
response to training decreases with
age42. We are aware
that competence in evidence-based medicine is not a technical skill. Numerous
reports have also shown differences in memory skills between young and older
healthy adults43.
Another factor could be the lack of computer utilization among older surgeons.
In a survey among members of the American Society of Internal Medicine,
younger physicians used computers more frequently in their clinical
work44. However,
lack of Internet facilities could not be a barrier since nearly all
respondents reported an ability to access the Internet at home (98%), in their
offices (98%), and in their local medical libraries (96%), and members of the
Dutch Orthopaedic Association have free access to The Journal's
full-text articles through the Association's web site.
Implications of the Study
Our study highlights surgeons' awareness of The Journal's
evidence-based medicine section and the Cochrane reviews. Furthermore, Dutch
orthopaedic surgeons welcome the emphasis on evidence-based medicine. The
Journal's decision to provide evidence summaries is largely vindicated by
our findings that respondents perceived these as the best way to move from
opinion-based practice to evidence-based practice.
Although awareness of evidence-based medicine was apparent, more than 50%
of the respondents thought that surgeons could be blinded in a randomized
trial of a surgical intervention. Clearly, more effort and education are
required to improve surgeon understanding in this respect. The
Journal's ongoing educational efforts are needed to change the paradigm
of orthopaedics toward an evidence-based
practice3,45-48.
The conclusion of a recent article by Hurwitz et
al.45 in the
Orthopaedic Forum section of The Journal, which stated that
"evidence-based practice has the potential to change the culture of
orthopaedics for the better and improve the quality of care for
patients," is strengthened by the findings of our survey. The welcoming
attitude of Dutch orthopaedic surgeons warrants further promotion and
implementation of evidence-based practice by the Dutch Orthopaedic
Association.
Unanswered Questions and Future Research
Our competence instrument needs to be validated in more detail, through
reliability and responsiveness testing. At the present time, most studies
evaluating knowledge about evidence-based medicine among physicians focus on
nonsurgical specialties. The findings of our study reflect the attitudes and
competence of orthopaedic surgeons in one European country and should be
repeated in other countries. Furthermore, we need to repeat our survey in the
Netherlands in the future to evaluate whether the attitudes and competence of
surgeons change over time.
In conclusion, evidence-based medicine is welcomed by Dutch orthopaedic
surgeons. The recent emphasis on evidence-based medicine is reflected in an
increased awareness about The Journal's evidence-based medicine
section and levels of evidence and the largest evidence-based medicine
resource, the Cochrane reviews. Younger orthopaedic surgeons had better
knowledge about evidence-based medicine. The development and use of
evidence-based guidelines as well as preappraised summaries like The
Journal's evidence-based medicine abstracts and the Cochrane reviews are
perceived as the best way to move from opinion-based to evidence-based
orthopaedic practice.
The questionnaire, the competence instrument coding scheme, the demographic
data on the respondents, the results of the univariable analysis related to
the competence instrument score, and the comparison of competence instrument
scores with use of evidence-based medicine resources are available with the
electronic versions of this article, on our web site at
(go to
the article citation and click on "Supplementary Material") and on
our quarterly CD-ROM (call our subscription department, at 781-449-9780, to
order the CD-ROM).
Bhandari M, Tornetta P 3rd.
Evidence-based orthopaedics: a paradigm shift. Clin Orthop Relat
Res. 2003;413:
9-10.4139
2003
[PubMed]
Poolman RW, Burgmans MC, Ultee JM.
"State of the art" behandeling van comminutieve
femur-schactfracturen: "evidence-based case report".
Nederlands Tijdschrift voor Traumatologie. 2002;
168-75.168
2002
Bhandari M, Guyatt GH, Montor V,
Devereaux PJ, Swiontkowski MF. User's guide to the orthopaedic literature: how
to use a systematic literature review. J Bone Joint Surg Am.
2002;84:
1672-82.841672
2002
[PubMed]
Wright JG, Swiontkowski MF. Introducing
a new journal section: evidence-based orthopaedics. J Bone Joint Surg
Am. 2000;82:
759.82759
2000
Wright JG, Swiontkowski MF, Heckman JD.
Introducing levels of evidence to the journal. J Bone Joint Surg
Am. 2003;85:
1-3.851
2003
[CrossRef]
Navarro FA. [Medical language based on
literature references of original articles in the Nederlands Tijdschrift voor
Geneeskunde (1930-1995)]. Ned. Tijdschr. Geneeskd.1996;140: 1263-7.
Dutch.1401263
1996
[PubMed]
Sauerland S, Lefering R, Neugebauer EA.
The pros and cons of evidence-based surgery. Langenbecks Arch
Surg. 1999;384:
423-31.384423
1999
[CrossRef]
Oliveri RS, Gluud C, Wille-Jorgensen PA.
Hospital doctors' self-rated skills in and use of evidence-based
medicine—a questionnaire survey. J Eval Clin Prac.
2004;10:
219-26.10219
2004
[CrossRef]
McColl A, Smith H, White P, Field J.
General practitioners' perceptions of the route to evidence based medicine: a
questionnaire survey. BMJ.
1998;316:
361-5.316361
1998
[PubMed]
Hanson BP, Bhandari M, Audige L, Helfet
D. The need for education in evidence-based orthopedics: an international
survey of AO course participants. Acta Orthop Scand.
2004;75:
328-32.75328
2004
[PubMed][CrossRef]
Man JP, Weinkauf JG, Tsang M, Sin DD.
Why do some countries publish more than others? An international comparison of
research funding, English proficiency and publication output in highly ranked
general medical journals. Eur J Epidemiol.
2004;19:
811-7.19811
2004
[PubMed][CrossRef]
Veenemans J, Hoogeveen J, Overbeke AJ.
[Dutch authors in international medical journals are also authors of
publications in the Nederlands Tijdschrift voor Geneeskunde (Dutch Journal of
Medicine)]. Ned Tijdschr Geneeskd.
2005;149: 2573-6.
Dutch.1492573
2005
[PubMed]
Sackett DL, Straus SE, Richardson WS,
Rosenberg W, Haynes RB. Evidence-based medicine: how to practice and
teach EBM. 2nd ed. New York: Churchill Livingstone;
2000.
2000
Edwards P, Roberts I, Clarke M,
DiGuiseppi C, Pratap S, Wentz R, Kwan I, Cooper R. Methods to increase
response rates to postal questionnaires. Cochrane Database Methodology
Rev. 2003;4:
MR000008.4MR000008
2003
Bland JM, Altman DG. Statistics notes:
Multiple significance tests: the Bonferroni method. BMJ.
1995;310:
170.310170
1995
[PubMed]
Seago JA, Spetz J, Keane D, Grumbach K.
College students' perceptions of nursing: a GEE approach. Can J Nurs
Leadersh. 2006;19:
56-74.1956
2006
Gibson ES, Powles AC, Thabane L, O'Brien
S, Molnar DS, Trajanovic N, Ogilvie R, Shapiro C, Yan M, Chilcott-Tanser L.
"Sleepiness" is serious in adolescence: two surveys of 3235
Canadian students. BMC Public Health.
2006;6:
116.6116
2006
[PubMed][CrossRef]
Carriere KC, Roos LL, Dover DC. Across
time and space: variations in hospital use during Canadian health reform.
Health Serv Res. 2000;35:
467-87.35467
2000
[PubMed]
Evans S, Li L. A comparison of goodness
of fit tests for the logistic GEE model. Stat Med.
2005;24:
1245-61.241245
2005
[PubMed][CrossRef]
Zeger SL, Liang KY. Longitudinal data
analysis for discrete and continuous outcomes. Biometrics.
1986;42:
121-30.42121
1986
[PubMed][CrossRef]
Horton NJ, Bebchuk JD, Jones CL, Lipsitz
SR, Catalano PJ, Zahner GE, Fitzmaurice GM. Goodness-of-fit for GEE: an
example with mental health service utilization. Stat Med.
1999;18:
213-22.18213
1999
[PubMed][CrossRef]
Horton NJ, Lipsitz SR. Review of
software to fit generalized estimating equation regression models. Am
Stat. 1999;53:
160-9.53160
1999
[CrossRef]
Goldhahn S, Audige L, Helfet DL, Hanson
B. Pathways to evidence-based knowledge in orthopaedic surgery: an
international survey of AO course participants. Int Orthop.
2005;29:
59-64.2959
2005
[PubMed][CrossRef]
Young JM, Glasziou P, Ward JE. General
practitioners' self ratings of skills in evidence based medicine: validation
study. BMJ. 2002;324:
950-1.324950
2002
[PubMed][CrossRef]
Jadad AR, Cook DJ, Jones A, Klassen TP,
Tugwell P, Moher M, Moher D. Methodology and reports of systematic reviews and
meta-analyses: a comparison of Cochrane reviews with articles published in
paper-based journals. JAMA.
1998;280:
278-80.280278
1998
[PubMed][CrossRef]
Moja LP, Telaro E, D'Amico R, Moschetti
I, Coe L, Liberati A. Assessment of methodological quality of primary studies
by systematic reviews: results of the metaquality cross sectional study.
BMJ. 2005;330:
1053.3301053
2005
[PubMed][CrossRef]
Gillespie WJ, Handoll H. Evidence based
medicine: the Cochrane Collaboration. Br J Sports Med.
1997;31:
172.31172
1997
[PubMed][CrossRef]
Gillespie WJ, Gillespie LD, Handoll HH,
Madhok R. The Cochrane Musculoskeletal Injuries Group. Acta Orthop
Scand Suppl. 2002;73:
15-9.7315
2002
Sur RL, Scales CD Jr, Preminger GM, Dahm
P. Evidence-based medicine: a survey of American Uro-logical Association
members. J Urol. 2006;176:
1127-34.1761127
2006
[PubMed][CrossRef]
Ramos KD, Schafer S, Tracz SM.
Validation of the Fresno test of competence in evidence based medicine.
BMJ. 2003;326:
319-21.326319
2003
[PubMed][CrossRef]
Johnston JM, Leung GM, Fielding R, Tin
KY, Ho LM. The development and validation of a knowledge, attitude and
behaviour questionnaire to assess undergraduate evidence-based practice
teaching and learning. Med Educ.
2003;37:
992-1000.37992
2003
[PubMed][CrossRef]
Boutron I, Moher D, Tugwell P, Giraudeau
B, Poiraudeau S, Nizard R, Ravaud P. A checklist to evaluate a report of a
nonpharmacological trial (CLEAR NPT) was developed using consensus. J
Clin Epidemiol. 2005;58:
1233-40.581233
2005
[CrossRef]
Boutron I, Tubach F, Giraudeau B, Ravaud
P. Blinding was judged more difficult to achieve and maintain in
nonpharmacologic than pharmacologic trials. J Clin Epidemiol.
2004;57:
543-50.57543
2004
[PubMed][CrossRef]
Boutron I, Tubach F, Giraudeau B, Ravaud
P. Methodological differences in clinical trials evaluating nonpharmacological
and pharmacological treatments of hip and knee osteoarthritis.
JAMA. 2003;290:
1062-70.2901062
2003
[PubMed][CrossRef]
Lilford R, Braunholtz D, Harris J, Gill
T. Trials in surgery. Br J Surg.
2004;91:
6-16.916
2004
[PubMed][CrossRef]
Devereaux PJ, Manns BJ, Ghali WA, Quan
H, Lacchetti C, Montori VM, Bhandari M, Guyatt GH. Physician interpretations
and textbook definitions of blinding terminology in randomized controlled
trials. JAMA. 2001;285:
2000-3.2852000
2001
[PubMed][CrossRef]
McAlister FA, Graham I, Karr GW,
Laupacis A. Evidence-based medicine and the practicing clinician. J Gen
Intern Med. 1999;14:
236-42.14236
1999
[CrossRef]
Burkiewicz JS, Zgarrick DP.
Evidence-based practice by pharmacists: utilization and barriers. Ann
Pharmacother. 2005;39:
1214-9.391214
2005
[CrossRef]
Obertop H. How surgeons make decisions:
authority and evidence. Ann Surg.
2005;242:
753-6.242753
2005
[PubMed][CrossRef]
Obertop H, van de Velde CJ. One hundred
years of surgical science behind the dikes. Br J Surg.
2002;89:
673-5.89673
2002
[PubMed][CrossRef]
Neumayer LA, Gawande AA, Wang J,
Giobbie-Hurder A, Itani KM, Fitzgibbons RJ Jr, Reda D, Jonas-son O; CSP #456
Investigators. Proficiency of surgeons in inguinal hernia repair: effect of
experience and age. Ann Surg.
2005;242:
344-8.242344
2005
[PubMed]
Sawaki L, Yaseen Z, Kopylev L, Cohen LG.
Age-dependent changes in the ability to encode a novel elementary motor
memory. Ann Neurol.
2003;53:
521-4.53521
2003
[PubMed][CrossRef]
Grady CL, McIntosh AR, Craik FI.
Task-related activity in prefrontal cortex and its relation to recognition
memory performance in young and old adults. Neuropsychologia.
2005;43:
1466-81.431466
2005
[PubMed][CrossRef]
Lacher D, Nelson E, Bylsma W, Spena R.
Computer use and needs of internists: a survey of members of the American
College of Physicians-American Society of Internal Medicine. Proc AMIA
Symp. 2000: 453-6.453
2000
Hurwitz SR, Tornetta P 3rd, Wright JG.
An AOA critical issue; how to read the literature to change your practice: an
evidence-based medicine approach. J Bone Joint Surg Am.
2006;88:
1873-9.881873
2006
[PubMed][CrossRef]
Bhandari M, Montori VM, Swiontkowski MF,
Guyatt GH. User's guide to the surgical literature: how to use an article
about a diagnostic test. J Bone Joint Surg Am.
2003;85:
1133-40.851133
2003
[PubMed]
Bhandari M, Guyatt GH, Swiontkowski MF.
User's guide to the orthopaedic literature: how to use an article about
prognosis. J Bone Joint Surg Am.
2001;83:
1555-64.831555
2001
[PubMed]
Bhandari M, Guyatt GH, Swiontkowski MF.
User's guide to the orthopaedic literature: how to use an article about a
surgical therapy. J Bone Joint Surg Am.
2001;83:
916-26.83916
2001
[PubMed][CrossRef]