We previously reported the results of a study in which a basic competency
examination in musculoskeletal medicine was administered to a group
of recent medical school graduates. This examination was validated
by 124 orthopaedic program directors, and a passing grade of 73.1%
was established. According to that criterion, 82% of the examinees
failed to demonstrate basic competency in musculoskeletal medicine. It
was suggested that perhaps a different passing grade would have
been set by program directors of internal medicine departments.
To test that hypothesis, and to determine whether the importance
of the individual questions would be rated similarly, the validation
process was repeated with program directors of internal medicine
residency departments as subjects.
Our basic competency examination was sent to all 417 program directors
of internal medicine departments in the United States. Each recipient
was mailed a letter of introduction explaining the purpose of the
study, a copy of the examination, and our answer key and scoring
guide. There was no mention of the results of the first study. The
subjects were requested to rate the importance of each question
on the same visual analog scale, ranging from "not important" to
"very important," as had been used by the orthopaedic program directors.
These ratings were converted into numerical scores. The program
directors were also asked to suggest a passing score for the examination,
and this score was used to assess the examinees' performance on
the examination. The results on the basis of the internal medicine
program directors' responses and those according to the orthopaedic
program directors' reponses were compared.
Two hundred and forty (58%) of the 417 program directors of internal
medicine residency departments responded. They suggested a mean
passing score (and standard deviation) of 70.0% ±; 9.9%.
As reported previously, the mean test score of the eighty-five examinees
was 59.6%. Sixty-six (78%) of them failed to demonstrate basic competency
on the examination according to the criterion set by the internal
medicine program directors. The internal medicine program directors
assigned a mean importance score of 7.4 (of 10) to the questions
on the examination compared with a mean score of 7.0 assigned by
the orthopaedic program directors. The internal medicine program
directors gave twenty-four of the twenty-five questions an importance
score of at least 5 and seventeen of the twenty-five questions an
importance score of at least 6.6.
According to the standard suggested by the program directors of
internal medicine residency departments, a large majority of the
examinees once again failed to demonstrate basic competency in musculoskeletal
medicine on the examination. It is therefore reasonable to conclude
that medical school preparation in musculoskeletal medicine is inadequate.