Carpal tunnel syndrome was chosen as the topic because of its
prevalence. In addition, it is a focused topic (unlike, for example,
arthritis or wrist fracture) and is thus amenable to computerized key-word
searching. The present study was designed to recreate the mechanism
by which individuals obtain information from the Internet: namely,
by identifying web sites through search engines. By entering the
desired phrase into a search engine, a collection of web sites containing relevant
information was identified. These web sites then were accessed and
evaluated.
Identification of Web Sites
The search phrase "carpal tunnel syndrome" was entered into the
five most commonly utilized World Wide Web search engines. A search
engine is a freely available computer program that allows a user
to scan the World Wide Web to find web pages that relate to the
entered search phrase. Search engines render the results of each
inquiry as a list of web-site addresses, or universal resource locators
(URLs). Search engines typically arrange URLs in decreasing order
of relevance to the search phrase. (Golladay et al.1 provide an excellent review of the
World Wide Web and its use.)
The five search engines that we used were Yahoo (www.yahoo. com),
Microsoft Network (www.msn.com), Netscape (www.netscape. com), Go/Infoseek
(www.go.com), and Lycos (www.lycos.com). These search engines were
selected because they were identified as the most commonly used
search engines at the time that this investigation was performed5. The top (first) fifty URLs listed
by each search engine were evaluated. These were combined to create
a master roster of 250 addresses.
Web-Site Evaluation
Each web site was evaluated according to the medical web-site
evaluation guidelines described by Soot et al.6.
The author, the nature of the web-site content, and the informational
value (defined below) of each web site were separately categorized
by two hand surgeons, and the results were assessed for reliability.
When the subjective characterization of the author or the nature
of the content differed between the two evaluators, a third evaluator
cast the deciding vote. The informational value of each site was
calculated by averaging the score of the two independent evaluators.
An aggregate profile of web sources on carpal tunnel syndrome was
thus defined.
Authorship
The author of each web site was assigned to one of seven categories:
(1) academic indicated that the author or authors
had a stated affiliation with a university or research organization; (2) physician indicated
that the author or authors were individual or group-practice physicians
who were not affiliated with a university or research organization
or whose affiliation was not stated on the web page; (3) nonphysician
care provider indicated chiropractors, physical and occupational therapists,
acupuncturists, and other alternative medical providers; (4) commercial
site indicated that the author represented a commercial web
site without an interest in a specific commercial product (typically,
the stated purpose of these web sites was to provide medical information);
(5) commercial product indicated an author or authors
who were marketing a commercial product for evaluation or treatment
of carpal tunnel syndrome; (6) lay indicated individuals
or organizations who did not belong to any of the previous categories
and who maintained a noncommercial web site for providing information
about carpal tunnel syndrome; or (7) unidentified indicated
that the author was not specified.
Content
The nature of the information regarding evaluation, treatment,
pathogenesis, and prevention of carpal tunnel syndrome on each site
was described according to one of four categories: (1) conventional indicated
that the site was dedicated to providing information consistent
with conventional knowledge as outlined in textbooks and orthopaedic
literature; (2) unconventional indicated that the
site provided alternative information in addition to conventional
knowledge without secondary commercial gains; (3) misleading indicated
that the site offered unconventional information with secondary
commercial gains; or (4) noninformational indicated
that the site was without patient-related information.
Informational Value
The informational value of each web site was measured according
to a scoring method with a maximum score of 100 points. A maximum
of 30 points was given for a complete disease summary; a maximum
of 20 points, for a complete review of the treatment options; a
maximum of 20 points, for a complete discussion of the pathogenesis
of carpal tunnel syndrome; and a maximum of 15 points each, for
a review of the complications and the results of treatment. The
points were assigned according to the following standards.
Disease summary (maximum, 30 points): Three
points each were awarded when any of the following ten factors were
mentioned: pain, weakness, numbness, anatomical distribution of
the median nerve, anatomy of the carpal tunnel, symptoms occurring
at night, decreased strength on physical examination, decreased
sensation on physical examination, provocative maneuvers on physical
examination (the Tinel sign, the Phalen sign, or the carpal compression
test), and diagnosis with nerve-conduction studies.
Treatment options (maximum, 20 points): Four
points each were awarded when any of the following treatment options
were given: splinting, oral anti-inflammatory medications, and corticosteroid
injections. Two points each were awarded if open carpal-tunnel release
or endoscopic carpal-tunnel release was given as the surgical treatment option.
Two points each (to a maximum of 4 points) were awarded for each
ergonomic tip (such as taking frequent breaks, modifying the setup
of the workstation, or changing body position) that was provided.
Pathogenesis (maximum, 20 points): Two points
were awarded for each of the following etiologies mentioned: a mass
in the carpal canal, an aberrant muscle, a hematoma, a wrist fracture,
diabetes mellitus, alcoholic neuropathy, hypothyroidism, rheumatoid
arthritis, gout, pregnancy, hemodialysis, obesity, and repetitive
stress.
Complications of treatment (maximum, 15 points): 7.5
points were awarded for each of the following categories mentioned:
complications of nonoperative treatment (such as progression of
neuropathy, side effects of oral anti-inflammatory medication, or
effects of cortisone injections) and complications of operative
treatment (such as pillar pain, infection, or nerve injury). A single
mention of a complication in each category was sufficient to earn
a full 7.5 points.
Results of treatment (maximum, 15 points): 7.5
points were awarded when the results of nonoperative treatment were
given, and 7.5 points were awarded when the results of operative
treatment were given. Again, a single mention of results in each
category was sufficient to earn a full 7.5 points.
Of the 250 web sites (the first fifty sites identified by five
search engines), 175 had a unique URL address and seventy-five were
duplications. Not one web site was identified by all five search
engines. Only two sites were listed by four of the five search engines.
The 175 unique web sites were analyzed with respect to authorship:
fifty-seven (33 percent) were characterized as a commercial product;
fifty-two (30 percent), as a commercial site; twenty-two (13 percent),
as an academic organization; seventeen (10 percent), as physicians;
sixteen (9 percent), as nonphysician care providers; nine (5 percent),
as unidentified; and two (1 percent), as laypersons. The two evaluators
agreed on the categorization of the authorship for 173 of the 175
sites, for an overall agreement rate of 99 percent.
The content of the web sites was assessed by the two evaluators.
They found that eighty sites (46 percent) provided conventional
information, fifty-four (31 percent) were noninformational, twenty-five
(14 percent) were misleading, and sixteen (9 percent) presented
unconventional information. The two evaluators agreed about their
content assessments for all 175 sites.
The informational value scores were calculated for all of the
web sites by both evaluators, and the two scores were averaged for
each site. The mean score for the 175 unique web sites was 28.4
of a maximum of 100 points, with a standard deviation of 28.3 points.
With the fifty-four noninformational sites excluded, the mean informational
value score for the remaining 121 sites was 41.1 points. The informational
scores for the web sites grouped by type of author are given in Table I.
The intraobserver reliability of the informational score assignment
was assessed. The mean difference in scores between the two evaluators
was 1.8 points. In 98 percent of the observations the scores assigned
by the two observers were within 8 points of each other, and in
87 percent the scores were within 5 points of each other, which
represents extremely high reliability.
Our review of the first fifty sites from five prominent search
engines returned 175 unique URL addresses. Almost two-thirds of
them pointed to a commercial site, and fewer than half of the web sites
offered conventional information. The mean informational value score
was 28.4 points for the 175 web sites. From these data, we concluded
that the quality of information available on the World Wide Web
is dubious. When users consult search engines to find web sites
and then visit the typical sites identified, they are unlikely to
encounter complete, unbiased, and conventional information. Because
the Internet has become a common source of information for patients,
this finding has practical importance to treating physicians.
The quality of information on the Internet for nonorthopaedic
medical conditions has been assessed, but we found no similar assessments
for orthopaedic conditions. In a letter published in the Journal
of the American Medical Association, Rose et al.4 reported on their study of web sites
identified by searching for orthopaedic conditions with use of terms
suggested by their patients. They found that only 20 percent of
the sites contained patient information and only 7 percent had information
that they deemed relevant.
The problem that both patients and physicians encounter is not
a lack of information but rather an overload of information, with
the valid and valuable information perhaps obscured by the oceans of
irrelevant and misleading information. With the increasing role
of the Internet as a main source of medical information for patients,
it can be assumed that patients are probably misinformed or, at
the least, distracted.
We contend that it is not practical - and, indeed, not possible
- for physicians to anticipate every piece of information that a
patient may have obtained prior to an office visit. This goes beyond
the sheer information glut on the web; one cannot even anticipate
the so-called top sources of information. In our study, there was
little consensus among the most commonly used search engines as to
what constituted a top site. Indeed, there were only seventy-five
duplications on the list of 250 sites, and none of the sites were
identified by all five search engines. This implies that it is likely that
information is gathered from a wide variety of sources. Accordingly,
it may not be reasonable to ask physicians to keep abreast of what
their patients may or may not know from Internet searches.
There were a few limitations to our study. To start, we based
our study on the web sites provided by popular search engines and
not on rosters of web sites listed according to overall usage by
people interested in carpal tunnel syndrome. It may well be the
case that even though 175 web sites were identified by the search
engines, most web users interested in carpal tunnel syndrome actually
visit only several of these sites. However, we believe that our
method of using search-engine results is reasonable as there are
few reliable lists of web sites based on usage, and even those would
not discern between general visits and visits for the specific purpose
of finding information on carpal tunnel syndrome.
The present study was not designed to rate the overall value
of the web postings; value, after all, is a subjective quality and
includes many features beyond the quality of information. Rather,
we aimed to measure objective attributes of popular web sites that
provide information about carpal tunnel syndrome. These attributes
include the source of information, with specific notation of the presence
of a commercial message in the web posting; whether the information
was conventional; and the completeness of the information relative
to a standard.
Our investigation was limited to the topic of carpal tunnel syndrome.
It may well be that other medical topics are covered more completely
and with less misleading information. It would not be correct to conclude
from this study that all orthopaedic medical information is of the
same caliber.
We believe that our measure of informational value is useful,
but it is clearly not the only metric for evaluating a web site.
This score is a measure of completeness, not accuracy. If a site
mentioned all of the factors that we looked for, it received a score of
100 points, even if there was also a sea of extraneous information.
A site that had only true statements, but very few of them, received
a low score. Thus, informational value does not account for overall
accuracy; instead, it measures the contribution of a single web
site to the complete education of the user. This measure was chosen
because we believe that completeness is an important feature. Visiting
a web site requires a substantial investment of time, and a user
may visit only a few sites in the course of searching for information.
The potential for the dissemination of inadequate or misleading
health-care information on the Internet has been recognized by some
individuals and organizations. For example, an Internet-based not-for-profit
organization, Health on the Net Foundation, which evaluates health-related
sites, has established guidelines in an attempt to standardize the
reliability and credibility of medical information on the World
Wide Web2. The principles established
by this group constitute a code of conduct to which web-site developers
should adhere in order to improve the quality of health-care information.
However, because the process of posting information on the World
Wide Web is unregulated, it remains unclear whether this and similar
organizations will have any significant impact on the quality of
available medical information.
In summary, our study demonstrated that the Internet offers a
wide variety of informational sources on carpal tunnel syndrome.
Although we may not have proven the contention that "myth, bias
and deception abound on the information superhighway,"3 we did indeed discover that the quality
of information regarding carpal tunnel syndrome is limited. Most
sites were not confined to conventional information, and even those
that presented only conventional information did not provide very much
of it.
We believe that the Internet poses an interesting challenge to
orthopaedic surgeons in that it arms patients with more information
than they had in the past. Surgeons must be vigorous in responding to
this challenge. To start, physicians must anticipate that patients
may have received bad information. Thus, an office visit should
include an open-ended question about what the patient thinks that he
or she knows, and time should be devoted to disabusing him or her
of any myths or errors. It is also reasonable to expect surgeons
themselves to become sources of high-quality information either by
posting web sites of their own or by publicizing sites known to
be of high quality. Finally, physicians must help patients to evaluate
the quality of the information that they encounter by educating them
with regard to web-site authorship and to potential conflicts of
interest associated with the information provided.