There were 396 applicants to our residency training program during the 2005
and 2006 application periods. Only declared published journal articles listed
on the Common Application Forms of the applicants were searched. One hundred
and forty-two applicants (35.9%) reported a total of 304 publications on their
forms. These included thirty-four citations listed as in print or in press,
twenty-eight citations to journals not listed in Ulrich's Periodicals
Directory, twenty-three citations to book chapters, and eighty-eight
citations listed as submitted. These 173 works were excluded from our
analysis. One hundred and thirty-one articles were declared from 103
applicants as appearing in journals listed in Ulrich's Periodicals
Directory, and we were able to verify all 131 of these papers per our
search methodologies.
After all 131 citations were checked and verified, twenty-seven or 20.6%
(95% confidence interval, 14.2% to 28.7%) were considered to be
misrepresented. Misrepresentation was defined as either (1) nonauthorship of
an existing article or (2) claimed authorship of a nonexistent article.
Thus, 104 true and actual publications were noted. The twenty-seven
misrepresented citations came from twenty-two (21%) of the 103 applicants who
had listed citations appearing in journals in Ulrich's Periodicals
Directory. Of the twenty-seven misrepresented listings, twelve citations
(44%) had no authorship by the applicant on an existing article and fifteen
citations (56%) involved claimed authorship of a nonexistent article. Of the
104 citations that were verified, nineteen listed the applicant as either
first or second author. Furthermore, only thirty-three (32%) of 104 verified
references were on orthopaedic or musculoskeletal topics.
Erroneous representation and falsification of physician credentials are
welldocumented
phenomena8-10.
Moreover, misrepresentation and falsification by medical students and other
applicants seeking residency training positions have also been noted, with an
incidence ranging from 6% to
41%2,11-19.
Cohen-Gadol et
al.18 reported the
lowest misrepresentation rate (6% of 102 applicants) in their paper and
concluded that the more "honest and accurate" candidates were a
reflection of the applicants' "collective integrity in representing
their skills and competency to the public in their future profession."
Nevertheless, Cohen-Gadol et al. later acknowledged in their report that four
of the six applicants to their residency program with misrepresented
publications successfully matched into a residency program.
There are many potential reasons why misrepresented publications may be so
prevalent. In one of the initial papers identifying misrepresented publication
rates, Sekas and
Hutson11 stated
several possibilities. Although they had identified misrepresentation rates
among gastroenterology fellowship applicants in their report, we believe their
explanations are also relevant to orthopaedic surgery residency applicants.
First, they mentioned that an augmented bibliography may give an applicant
more of a competitive advantage during the residency selection process. Since
orthopaedic surgery residency programs are continuing to attract many more
applicants than there are available positions, competition to obtain a
residency position is manifest, and any advantage such as a stronger-appearing
publication record may help a candidate to rank higher in the selection
process. Second, some applicants may believe in a universal perception that
"everyone does it," and therefore an enhanced publication record
is a standard and common practice. Third, for some applicants, the benefits of
matching in their field of choice may outweigh the risks of being discovered.
Fourth, some applicants may not be aware that listing an article as being in
press is not the same as listing it as having been submitted or under review.
Finally, misrepresented publications may be a sign of an underlying
psychiatric condition of the applicant that could be associated with other
undesirable behaviors.
As mentioned above, in competitive specialties such as orthopaedic surgery,
applicants may be tempted to misrepresent their publications intentionally to
gain an advantage over their peers. Baker and
Jackson17 reported
a misrepresentation rate of 11% among seventy-three applicants to their
radiology residency program who had listed publications, and they commented
that candidates who are fraudulent in their residency applications might
continue to be fraudulent in other areas of their careers such as with
interactions among colleagues and patients.
However, other fields that are perhaps somewhat less competitive for
obtaining a residency position have also reported high rates of
misrepresentation. Roellig and
Katz16 reported a
misrepresentation rate of 21.3% (ten of forty-seven) for applicants to their
emergency medicine residency program who had listed publications, and they
felt that it was reasonable to assume that misrepresented publications may
indicate an unwanted behavioral pattern that would be completely unacceptable
in a medical setting. Bilge et
al.14 reported a
misrepresentation rate of 19.7% among the 147 applicants to their pediatrics
residency program who had listed publications, and those authors suggested
that the National Resident Matching Program use a centralized clearinghouse
for all residency program applications. Any candidate with an identified
misrepresented publication could then be prevented from applying to residency
programs. They further insist that if misrepresentation of publications
continues to be a problem, then residency selection committees should
reconsider the value of authorship of peer-reviewed publications in the
selection of future residents.
We hypothesized that our previous report and those of other investigators
would lead to a reduction in the proportion of misrepresented citations.
Instead, we found a slight increase in the misrepresentation rate (from 18% to
20.6%). There are several potential explanations for our findings. First, it
is possible that candidates made typographical or clerical errors when they
completed their Electronic Residency Application Service applications.
However, given our rigorous process utilizing the most comprehensive medical
database search engine, this seems unlikely. Also, given the applicants' level
of education, such gross errors seem improbable. Second, it is possible that
the listed citations for unpublished works were actually in press, submitted,
or in process. However, this would also require clerical errors, an unlikely
occurrence in this group of highly educated, computer-savvy applicants. Third,
one may surmise that the misrepresentations were intentional. By portraying
their research efforts as published, applicants might hope to garner a more
favorable evaluation by review committees and program directors.
Of the 304 publications claimed by orthopaedic surgery residency
applicants, including published articles from journals not listed in
Ulrich's Periodicals Directory as well as book chapters and articles
that were in press, in print, or submitted, only 104 (34.2%) were actually
found to be published in journals listed in Ulrich's Periodicals
Directory. Although published scholarly work may add bulk to an
applicant's curriculum vitae, residency programs may wonder whether citations
not in journals listed in Ulrich's Periodicals Directory are of any
scientific importance but also whether they exist at all. In addition to
perhaps being misrepresentations or fabricated references, these other
so-called publications occur in non-peer-reviewed sources (e.g., pamphlets or
newspapers) and are not easily accessible to clinicians, researchers, and
program directors.
The most frequent listings in the Publications section of the Common
Application Form were the citations labeled as submitted, in print, or in
press (122 or 40% of all citations). Since there was no way of verifying these
scholarly activities, they were not included in our analysis. Patel et
al.9 reviewed
articles listed on the curriculum vitae of 151 orthopaedic fellowship
applicants that were defined as accepted, in press, submitted, or in progress.
After a two-year period, it was found that the actual rate of publication was
only 4% (twenty-one of 532) in the combined category of papers submitted
and/or in progress. This result is not totally unexpected as only 46% (668) of
the 1465 scientific reports presented at three consecutive American Academy of
Orthopaedic Surgeons meetings were eventually published nearly four years
after
presentation20.
Our retrospective study has several potential limitations. First, despite
spanning a two-year period, our sample of residency applicants was relatively
small. Second, while the current study was conducted at a different
institution from that in our earlier report, our findings may not be
generalizable to other settings. Finally, it is possible that some of the
misrepresentation was due to unintentional errors in citing references. A
typographic or clerical error could make it difficult to verify a citation
properly. Indeed, one report analyzing misrepresentation among internal
medicine residents argued that previous studies may have overestimated the
magnitude of
misrepresentation19.
Those authors encouraged programs to contact coauthors and editors of journals
since previously reported searches were not adequately comprehensive. However,
in our study, all citations were searched with use of PubMed-MEDLINE Internet
search engines after the applicant's Common Application Form from his or her
Electronic Residency Application Service application had been scanned into a
Microsoft Word document. We are confident that this process minimized errors
by the investigator resulting from misspelling or miscopying of citations onto
the search engine. When a journal was not listed in PubMed-MEDLINE, an
exhaustive interlibrary search as well as review of the journal of alleged
publication was performed.
Because the application process for orthopaedic surgery residency programs
has been streamlined recently into a single computer-based, Internet-accessed
program (the Electronic Residency Application Service), applicants are limited
in the manner in which they are able to report research endeavors. Currently,
the Common Application Form for the Electronic Residency Application Service
provides space only for the entry of publications and not for other scholarly
activities (e.g., posters, presentations, or articles in press). Thus, to be
considered as having research accomplishments, some applicants may feel
pressured into presenting their research endeavors in publication format.
After careful review of the applications to our program, it appears that many
candidates failed to differentiate between published articles and other
scholarship such as presentations, posters, papers in progress, papers
submitted, and abstracts. Nearly all applicants using the Publications section
on the Common Application Form list all types of academic endeavors including
oral and poster presentations, current projects, and published articles. We
found that applicants list these other activities in standard bibliography
format. Thus, without careful scrutiny and monitoring, such listings may be
confused with published works of scholarship. Consequently, the applicant
appears to have more publications than is the case.
While the Electronic Residency Application Service program has streamlined
the residency application process, it may also provide a way for verification
of cited publications. Because the proper citation of publications requires a
journal title, volume, and page numbers, this information can be automatically
linked to a PubMed-MEDLINE unique identifier number that can be inserted as a
link into the Electronic Residency Application Service program. Thus, with
this extra step, not only would program directors be able to verify instantly
the authenticity of papers published in peer-reviewed journals but they could
also print the articles for use in the interview and selection process. In
addition, the creation of new entry fields in the Common Application Form to
include abstracts, posters, presentations, articles submitted, and articles in
press should greatly help to clarify any ambiguity about the representation of
research projects.
In medical school, academic accomplishments are important, although not
completely valid, predictors of resident
performance21,22.
Other criteria used to differentiate among applicants to an orthopaedic
surgery residency program include research experience and published
scholarship1,23.
Moreover, analyses by the American Academy of Orthopaedic Surgeons and the
credentialing committee of the American Board of Orthopaedic Surgery have
shown that problems with attending surgeon professionalism can be predicted by
unethical behavior during medical school and
residency24. Since
personal attributes developed prior to residency (i.e., integrity,
professionalism, and communication skills) predict resident behavior, methods
for assessing the personal characteristics and qualities of residents need to
be
improved1,25.
Simon24 proposed
the development of procedures to identify and eliminate individuals with
deficiencies in ethical conduct before residency. Other than the current
strategies of stressing professionalism and ethical standards in medical
school as well as carefully reviewing all resident applications, the
measurement of an applicant's character is indeed challenging. The
identification of misrepresented research publications should be part of the
process for ensuring that residency programs select future residents with the
highest level of professionalism.