The Journal of Bone & Joint Surgery

The Journal of Bone & Joint Surgery, Volume 90, Issue 1

The Orthopaedic Forum | January 01, 2008

Readability of Patient Education Materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America Web Sites

Sameer Badarudeen, MB, BS¹; Sanjeev Sabharwal, MD¹

¹ University of Medicine and Dentistry of New Jersey, New Jersey Medical School, Doctor's Office Center, 90 Bergen Street, Suite 7300, Newark, NJ 07103. E-mail address for S. Sabharwal: sabharsa@umdnj.edu

View Disclosures and Other Information

Disclosure: The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity. No commercial entity paid or directed, or agreed to pay or direct, any benefits to any research fund, foundation, division, center, clinical practice, or other charitable or nonprofit organization with which the authors, or a member of their immediate families, are affiliated or associated.

Investigation performed at the University of Medicine and Dentistry of New Jersey-New Jersey Medical School, Newark, New Jersey

The Journal of Bone and Joint Surgery, Inc.

J Bone Joint Surg Am, 2008 Jan 01;90(1):199-204. doi: 10.2106/JBJS.G.00347

5 Recommendations (Recommend) | 3 Comments | Saved by 3 Users Save Case

Article

Comments (2)

text A A A

Abstract

Background: While experts recommend that the readability of patient education materials should be less than the sixth grade level, the available information pertaining to orthopaedic diseases may be excessively complex for some to read and comprehend. The Flesch-Kincaid grade level is the most widely used tool to evaluate the readability score of a given text, with a lower grade level suggesting easier readability. The goal of our study was to assess the readability of pediatric orthopaedic patient education materials that were developed by the American Academy of Orthopaedic Surgeons (AAOS) and the Pediatric Orthopaedic Society of North America (POSNA) and were accessible to the general public through the Internet.

Materials and Methods: All articles from the "Children" section of the patient education library, "Your Orthopaedic Connection," on the AAOS web site and the "Parent/Patient" section on the POSNA web site were identified. The Flesch-Kincaid grade level of each article was determined with use of Microsoft Office Word software. The mean grade levels of articles that were available in 2001 were compared with those accessible in 2007.

Results: Fifty-seven unique articles were available in 2007 on both web sites compared with twenty-five articles available in 2001. The readability score of only one (2%) of the currently available articles was less than sixth grade level. The mean Flesch-Kincaid grade level of the currently available articles was 8.9 compared with 8.7 for the articles available in 2001 (p = 0.71).

Conclusions: Our findings suggest that most of the pediatric orthopaedic patient education materials available on the AAOS and POSNA web sites have readability scores that may be too high, making comprehension difficult for a substantial portion of the United States population.

Figures in this Article

Introduction

The Internet is fast becoming the preferred source for patients to acquire health information^1-7. Studies have shown that patients with musculoskeletal complaints frequently use the Internet to learn about their illness^1-3. Access to the Internet has also improved greatly over the last decade, especially in the United States. Brooks reported that 75% of patients attending an orthopaedic clinic had access to the Internet either themselves or through a close friend or family member⁴. In 2006, 113 million Americans searched online for health information and each day eight million Americans sought health information from the Internet⁵. The majority of the people who searched for health information on the Internet claimed that this information had an impact on their health-care decisions⁵.

However, the health information available on the Internet may not be easily understood by the average person. One of every four individuals who obtained health information from the Internet found the information "overwhelming."⁵ In a survey conducted at a pediatric orthopaedic clinic⁶, 96% of the respondents felt that there is "too much information" available on the Internet and 63% felt that the information available is "too technical." The ability to use the health information received from the Internet depends largely on the individual's ability to comprehend the material⁷.

Readability is a major factor that affects the ability of a person to understand health information^8-10. The National Cancer Institute defines readability as the determination by systematic formulae of the reading comprehension level a person must have to understand written materials⁸. Studies have shown that the readability and comprehension of a given text are closely related^9,10. Among the many methods used to test readability, the Flesch-Kincaid grade level and the Flesch Reading Ease score are the most commonly cited^9,10. The Flesch-Kincaid grade level of a written text indicates that a person with reading skills equivalent to that of someone who has graduated from that "academic grade" will be able to read and comprehend the given material. For example, if the Flesch-Kincaid grade level of an article is 12, it means that only someone who has reading skills equivalent to that of a high school graduate will be able to read and comprehend that text. The Flesch Reading Ease score indicates how easy it is to read a particular passage, and the possible score ranges from 0 (unreadable) to 100 (very easy to read)¹¹. Thus, the Flesch-Kincaid grade level and Flesch Reading Ease score are inversely related.

The reading and comprehensive ability of the target population is often overlooked when the quality of health education materials is assessed¹². A recent national adult literacy survey found that ninety-two million Americans have low (below eighth-grade-level) or very low (below fifth-grade-level) reading skills^13,14. One-fifth of Americans are unable to comprehend even fourth-grade-level materials⁹. Authors from various fields of medicine^8-10 have found that the disparity between the target audience's reading ability and the readability of the available written health education materials is widespread. However, a detailed analysis of patient education materials for orthopaedics is lacking¹⁵.

The goal of our study was to determine the readability of pediatric orthopaedic patient education materials that were developed by the American Academy of Orthopaedic Surgeons (AAOS) and the Pediatric Orthopaedic Society of North America (POSNA) and were available to the general public through their respective web sites. Our hypothesis was that the readability of such material would be a Flesch-Kincaid grade level of =6.

Materials and Methods

The study was initiated after receiving a waiver from our institutional review board. The "Children" section of the patient education library on the AAOS web site ()¹⁶, called "Your Orthopaedic Connection," and the "Parent/Patient Information" section of the POSNA web site ()¹⁷ were searched. The patient education material available in February 2007 on these web sites was downloaded directly from the Internet, while those articles which were available at the AAOS and POSNA web sites in 2001 were obtained from the Internet Archive database on the archive.org web site¹⁸. All of the patient education articles available at the AAOS and POSNA web sites were assessed for inclusion in the study. Articles were excluded if the material available was only in graphic or pictorial form, if the entire body of text was in table or list format, or if the article was written in a language other than English.

The available information for each topic was saved as individual Microsoft Word documents. The text was copied in plain text format to avoid HTML tags. Any text unrelated to the patient information topic, such as information related to web page navigation, copyright notice, disclaimers, date stamps, author information, source information, feedback questionnaires, and hyperlinks, was omitted. All tables, citations, web site uniform resource locators (URLs), address fields, and telephone numbers were deleted to avoid influencing the readability scores. Follow-up editing, as recommended by Flesch and others^11,19, was done to avoid underestimation of the difficulty in reading the text. Thus, all of the numbers, decimals, bullets, abbreviations, paragraph breaks, colons, semicolons, and dashes within a sentence were removed. The Flesch-Kincaid grade level and Flesch Reading Ease score were calculated with use of the latest available version of Microsoft Office Word software (version 2003 Service Pack 2; Microsoft, Redmond, Washington). Although this feature is built into the software, it is disabled by default and the user has to enable it by sequentially going to "Tools," "Options," "Spelling and Grammar," and then enabling the option "Show readability statistics."

Readability scores of each article were then assessed by checking the spelling and grammar with use of the "Spelling and Grammar" tool. Once the spelling and grammar checking is complete, the software displays the readability scores on the screen.

The underlying formula for manual calculation of the Flesch-Kincaid grade level^11,20 is as follows:(0.39?×?average?number?of?words?per?sentence)?+?(11.8?×?average?number?of?syllables?per?word)-15.59.

The formula for the Flesch Reading Ease score¹⁹ is as follows:206.835-(1.015?×?average?number?of?words?per?sentence)-(84.6?×?average?number?of?syllables?per?word).

Statistical Analysis

The intraclass correlation coefficient was used to assess intraobserver reliability for calculating the Flesch-Kincaid grade level by reassessing the grade level of fifteen randomly selected articles one week apart. An intraclass correlation coefficient of 0 to 0.24 indicates poor correlation; 0.25 to 0.49, low correlation; 0.50 to 0.69, fair correlation; 0.70 to 0.89, good correlation; and 0.90 to 1.0, excellent correlation²¹. All Flesch-Kincaid grade levels and Flesch Reading Ease scores were calculated by the same individual. The mean and 95% confidence intervals were calculated for both Flesch-Kincaid grade level and Flesch Reading Ease score. The Flesch-Kincaid grade level of currently available (February 2007) articles and that of the articles available in 2001 were compared with use of the two-sample t test. The level of significance was set with an alpha value of <0.05.

Results

The intraclass correlation coefficient for assessing Flesch-Kincaid grade level was 0.99, indicating excellent intraobserver reliability. A total of fifty-two articles from the AAOS web site¹⁶ and forty-nine articles from the POSNA web site¹⁷ were available on February 25, 2007. Of the fifty-two articles at the AAOS site, forty-one articles were the same as those available through the POSNA site. Thus, a combined total of sixty unique articles on various pediatric orthopaedic topics were available from both web sites. Of those sixty articles, two were excluded because the entire text was in table format and one article was excluded because the entire text was in list format. For the remaining fifty-seven articles, the mean Flesch-Kincaid grade level was found to be 8.9 (95% confidence interval, 8.5 to 9.2). The details of the articles available in February 2007, including the Flesch-Kincaid grade level, are listed in a table in the Appendix. Only one article, "Exercises for Young Athletes," with a readability grade level of 3.6 was found to be at or below the sixth grade level, the recommended readability level appropriate for the adult patient in the United States⁹. Forty-one articles (72%) had a readability level above eighth grade, and this would be beyond the mean reading ability level of the U.S. adult population⁹ (Fig. 1).

View Large | Download Slide(.ppt)
Add to My JBJS

Fig. 1: Patient education articles developed by the Pediatric Orthopaedic Society of North America (POSNA) and the American Academy of Orthopaedic Surgeons (AAOS) appropriate for various Flesch-Kincaid (F-K) grade levels. For example, if the reading ability of the target population is grade 8, then only 28% of the fifty-seven articles on the POSNA and AAOS web sites will be appropriate for them. Note that the reading level of the typical American patient is between grades 4 and 6.

The mean Flesch Reading Ease score was 57.3 (95% confidence interval, 55.3 to 59.2). The readability grade level and reading ease scores of articles listed on the POSNA web site were not compared with those available at the AAOS site, since all of the articles available through POSNA were the same articles that were available through AAOS and thus had the same readability.

None of the articles available in February 2007 on either web site were available in 2001. Eighteen articles in the pediatric section of the patient education library of the AAOS web site and seven articles on the POSNA web site were available in 2001. For the twenty-five articles available in 2001, the mean Flesch-Kincaid grade level was 8.7 (95% confidence interval, 8.3 to 9.2). The mean Flesch-Kincaid grade levels of the twenty-five articles available in 2001 and the fifty-seven articles available in 2007 were not significantly different (p = 0.71).

Discussion

Health literacy is defined as the "degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions."²² Health literacy and health outcomes are closely related^23,24. Improved health literacy is associated with better health outcomes^23,25-27. Health literacy is the single best predictor of an individual's health status²⁸. Low health literacy is associated with poor general health, an increased rate of hospitalization, increased complications, more missed appointments, noncompliance with treatment, poor understanding of one's disease, and an overall increase in health-care expenditure^9,29. A national survey conducted recently found that one of every two adult Americans has difficulty understanding and taking action on health information^22,23.

Our study provides some insight as to why consumers may be finding it difficult to understand the available patient information material on the Internet. The readability of patient education articles on the two web sites we studied is comparable with findings from studies done in other fields of medicine^7,9,12,30-35. Various health organizations, such as the National Work Group on Cancer and Health³⁶, the American Medical Association²⁸, and the National Institutes of Health³⁷, recommend that the readability of patient information material be less than a sixth grade level, while the Centers for Disease Control and Prevention recommends the readability to be between the sixth and seventh grade levels³⁸. We found that only one of the fifty-seven available online pediatric orthopaedic patient education articles available at the AAOS and POSNA web sites had the recommended readability grade level. These findings highlight the need for clinicians and other health-care workers to recognize the concept of readability in order to improve the understanding of the disease process and various treatment options by the patients and their caretakers.

Using a relatively simple set of keystroke instructions available on most personal computers, one can easily assess the readability level of a given text. The fact that there was at least one article in our analysis with readability of less than sixth grade level shows that creating patient education materials on health information topics with readability appropriate for people with limited reading skills is an attainable target. However, our finding that the readability of available articles had not substantially changed between the years 2001 and 2007 shows the lack of awareness of this issue in the orthopaedic community.

When patient information materials are prepared, the reading skills of the target population should be taken into consideration. A study in 1991 found that patients read at a level about 4.6 grades lower than their highest attained academic grade, and their average reading grade was 5.4^32,39. Thus, the academic grade achieved should not be used as an indicator of an individual's reading skill. It has also been shown that the grade level achieved in school, ethnicity, socioeconomic class, attire, speech, appearance, or occupation cannot be reliably used to identify individuals with poor health literacy^9,39. Even self-reported ability to read well can be misleading^10,39-41. The data on the reading skills of Internet users are lacking. However, with increasing accessibility to and use of the Internet, people who search online for health information are becoming more representative of the U.S. population⁵. The finding that 33% of the "online health seekers" had only a high school diploma or less education⁵ adds further support to this assumption. Recently, a national survey found that 20% of adults with "below basic health literacy" are getting their health information from the Internet¹³. Studies have also shown that even people with good literacy skills prefer materials written in simpler format and a low grade level^9,28. In another study, 83% of the patients who used the Internet to get health information found the readability unacceptably high⁴². Furthermore, many experts are recommending printing handouts of articles from the Internet for patient education². Thus, the patient education information that is available online should be appropriate for comprehension by all sectors of society.

The readability of a given health information text may be improved by incorporating the following suggestions. Complex medical terms should be replaced by simpler terms. A list of lay terms that can be used instead of medical terms can be found on the following web sites: and ^36,38. Each sentence should be short and not contain more than eight to ten words³⁹. Comprehension of written health information can be further enhanced by the use of illustrations, good formatting, and better use of color, font type, and size⁸. There are some guidelines available on the appropriate use of diagrams and illustrations in patient education materials⁹. These suggestions include placing the diagrams and illustrations in close proximity to the relevant text, providing appropriate captions that explain the picture, highlighting the area of interest in the picture by arrows or encircling, and using line drawings, which are less distractive than color photos. Pictures should reduce the amount of text to read, and large pictures should be cropped in such a way that each picture conveys only one message. The above guidelines may be adopted to improve the readability of currently available articles in the AAOS and POSNA patient education libraries available online. It would also be helpful if the content in the patient education library is layered or stratified in such a manner that visitors can go from the most basic information to in-depth information about a disease using hyperlinks and navigation menus.

Although there have been some reports on the quality of orthopaedic patient education materials in the past^1-4,6,15, to our knowledge, no prior study has addressed the readability issue. The instrument we used to assess the readability, that is, the Flesch-Kincaid grade level, is widely available through the ubiquitous Microsoft Office software package. This readability assessment tool, originally developed for the U.S. Armed Forces, has been extensively validated and is relatively simple, quick, and intuitive^10,11,32. As seen in our study, the Flesch-Kincaid grade level has high intraobserver reliability. Furthermore, the Flesch-Kincaid scoring system has been shown to have excellent correlation with other available readability scores^11,43,44. There are several other software packages available for assessing the readability level of a given text, such as Corel WordPerfect Office X3 (Corel, Ottawa, Ontario, Canada), Readability Calculations (Micro Power and Light, Dallas, Texas), Readability Studio 1.1 (Oleander Solutions, Vandalia, Ohio), and Intext (Social Science Consulting, Rudolstadt, Germany)^37,45.

Although there is general agreement that readability influences comprehension^9,10, we did not assess the reading skills of our patient population. This is an important limitation of this study. We recognize that all readability formulas including the Flesch-Kincaid scores provide only indirect evidence of comprehensibility of the written material. Another drawback of our methodology is that the Flesch-Kincaid scores take into consideration only the written material in an article³⁶. Comprehension of a health education brochure can be enhanced by the inclusion of pictures and diagrams, which are not captured by the Flesch-Kincaid grade level or Flesch Reading Ease scores. There are other readability assessment tools, such as the Suitability Assessment of Materials^9,46-48, which can assess the effectiveness of pictures and diagrams as well as written texts. However, unlike the Flesch-Kincaid grading system, the Suitability Assessment of Materials is a relatively new instrument and has not been extensively validated, is more time consuming⁹ and "inherently subjective"⁴⁸, and thus may not be practical for general use. Another potential criticism of our study is that we reviewed online materials that were developed by the AAOS and POSNA only. While this subset of articles probably constitutes only a small proportion of the patient education materials available online, our sample is quite relevant, since many orthopaedic surgeons are directing their patients to such professional web sites to access up-to-date and peer-reviewed information^15,49,50.

The AAOS introduced the web-based patient education library in 2000, with the goal of strengthening the patient-physician relationship⁵¹. It was developed to "be sensitive to diversity and readability."⁵¹ Our findings suggest that most of the available pediatric orthopaedic patient education material available on the AAOS and POSNA web sites have readability scores that may be too difficult for comprehension by a substantial portion of the U.S. population. This issue should be addressed as new educational materials are developed for online dissemination. By improving the readability of patient education materials, orthopaedic surgeons can further improve their communication skills and positively impact the health outcome of their patients.

Appendix

A table listing the details of all articles for 2007 that were evaluated is available with the electronic versions of this article, on our web site at (go to the article citation and click on "Supplementary Material") and on our quarterly CD-ROM (call our subscription department, at 781-449-9780, to order the CD-ROM).

Acknowledgements

Note: The authors thank Emily McClemens, PA-C, for her technical assistance.

References

Beall MS 3rd, Golladay GJ, Greenfield ML, Hensinger RN, Biermann JS. Use of the Internet by pediatric orthopaedic outpatients. J Pediatr Orthop.2002;22:261-4.22261 2002 [PubMed][CrossRef]

Krempec J, Hall J, Biermann JS. Internet use by patients in orthopaedic surgery. Iowa Orthop J.2003;23:80-2.2380 2003

Beall MS 3rd, Beall MS Jr, Greenfield ML, Biermann JS. Patient Internet use in a community outpatient orthopaedic practice. Iowa Orthop J.2002;22:103-7.22103 2002

Brooks BA. Using the Internet for patient education. Orthop Nurs.2001;20:69-77.2069 2001

Fox S. Online Health Search 2006. Pew Internet and American Life Project. 2006 Oct 29. . Accessed 2007 Feb 6. 2006
http://www.pewinternet.org/pdfs/PIP_Online_Health_2006.pdf

Aslam N, Bowyer D, Wainwright A, Theologis T, Benson M. Evaluation of Internet use by paediatric orthopaedic outpatients and the quality of information available. J Pediatr Orthop B.2005;14:129-33.14129 2005 [CrossRef]

Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, Kanouse DE, Muñoz JA, Puyol JA, Lara M, Watkins KE, Yang H, McGlynn EA. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA.2001;285:2612-21.2852612 2001 [CrossRef]

Albright J, de Guzman C, Acebo P, Paiva D, Faulkner M, Swanson J. Readability of patient education materials: implications for clinical practice. Appl Nurs Res.1996;9:139-43.9139 1996 [CrossRef]

Doak CC, Doak LG, Root JH. Teaching patients with low literacy skills. 2nd ed. Philadelphia: J.B. Lippincott; 1996. 1996

Cooley ME, Moriarty H, Berger MS, Selm-Orr D, Coyle B, Short T. Patient literacy and the readability of written cancer educational materials. Oncol Nurs Forum.1995;22:1345-51.221345 1995

Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav.2006;33:352-73.33352 2006 [CrossRef]

Boulos MN. British internet-derived patient information on diabetes mellitus: is it readable? Diabetes Technol Ther.2005;7:528-35.7528 2005 [CrossRef]

Kutner M, Greenberg E, Baer J. National Assessment of Adult Literacy (NAAL). A first look at the literacy of America's adults in the 21st century. National Center for Education Statistics. 2005 Dec 15. . Accessed 2007 Feb 6. 2005
http://nces.ed.gov/NAAL/PDF/2006470.PDF

Kirsch IS, Jungeblut A, Jenkins L, Kolstad A. Adult literacy in America: a first look at the results of the National Adult Literacy Survey. National Center for Education Statistics. 1993 Sep. . Accessed 2007 Feb 6. 1993
http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/13/da/b3.pdf

Rozental TD, Bozentka DJ, Beredjiklian PK. Patient education through the Internet: academic and private practice sites. Clin Orthop Relat Res.2004;421:50-3.42150 2004 [CrossRef]

Richards BS III, editor. Your orthopaedic connection: children. American Academy of Orthopaedic Surgeons. . Accessed 2007 Feb 6. 2007
http://orthoinfo.aaos.org/menus/children.cfm

POSNA patient/parent information. The Pediatric Orthopaedic Society of North America. . Accessed 2007 Feb 6. 2007
http://www.posna.org/web/resources/patientInformation.html

Internet archive: wayback machine. 2001 Mar 10. . Accessed 2007 Feb 20. 2001
http://www.archive.org/web/web.php

Flesch RF. How to write plain English: a book for lawyers and consumers. New York: Barnes and Noble; 1981. 1981

Kincaid JP, Fishburne RP, Rogers RL, Chissom BS. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Springfield, VA: National Technical Information Service; 1975. 1975

Munro BH. Statistical methods for health care research. 5th ed. Philadelphia: Lippincott Williams and Wilkins; 2005. Correlation; p 239-58. 2005

Kutner M, Greenburg E, Jin Y, Paulsen C. The health literacy of America's adults: results from the 2003 National Assessment of Adult Literacy. National Center for Education Statistics. 2006 Sep. . Accessed 2007 Feb 20. 2003
http://nces.ed.gov/pubs2006/2006483.pdf

Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Health literacy: a prescription to end confusion. Washington, DC: National Academies Press; 2004. 2004

French KS, Larrabee JH. Relationships among educational material readability, client literacy, perceived beneficence, and perceived quality. J Nurs Care Qual.1999;13:68-82.1368 1999 [CrossRef]

Schillinger D, Grumbach K, Piette J, Wang F, Osmond D, Daher C, Palacios J, Sullivan GD, Bindman AB. Association of health literacy with diabetes outcomes. JAMA.2002;288:475-82.288475 2002 [CrossRef]

Dewalt DA, Berkman ND, Sheridan S, Lohr KN, Pignone MP. Literacy and health outcomes: a systematic review of the literature. J Gen Intern Med.2004;19:1228-39.191228 2004 [CrossRef]

Berkman ND, DeWalt DA, Pignone MP, Sheridan SL, Lohr KN, Lux L, Sutton SF, Swinson T, Bonito AJ. Literacy and health outcomes. Evidence report/technology assessment no. 87. Rockville, MD: Agency for Healthcare Research and Quality; 2004. 2004

Weiss BD. Health literacy: a manual for clinicians. Chicago, IL: American Medical Association Foundation; 2003. 2003

Baker DW, Parker RM, Williams MV, Clark WS, Nurss J. The relationship of patient reading ability to self-reported health and use of health services. Am J Public Health.1997;87:1027-30.871027 1997 [CrossRef]

Meade CD, Diekmann J, Thornhill DG. Readability of American Cancer Society patient education literature. Oncol Nurs Forum.1992;19:51-5.1951 1992

Meade CD, Byrd JC. Patient literacy and the readability of smoking education literature. Am J Public Health.1989;79:204-6.79204 1989 [CrossRef]

Ley P, Florio T. The use of readability formulas in health care. Psychol Health Med.1996;1:7-28.17 1996 [CrossRef]

D'Alessandro DM, Kingsley P, Johnson-West J. The readability of pediatric patient education materials on the World Wide Web. Arch Pediatr Adolesc Med.2001;155:807-12.155807 2001

Alexander RE. Readability of published dental educational materials. J Am Dent Assoc.2000;131:937-42.131937 2000

Davis TC, Mayeaux EJ, Fredrickson D, Bocchini JA Jr, Jackson RH, Murphy PW. Reading ability of parents compared with reading level of pediatric patient education materials. Pediatrics.1994;93:460-8.93460 1994

Cotugna N, Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education pages in health-related journals. J Community Health.2005;30:213-9.30213 2005 [CrossRef]

National Institutes of Health. How to write easy to read health materials. Updated 2007 Aug 22. . Accessed 2007 Nov 8. 2007
http://www.nlm.nih.gov/medlineplus/etr.html

Centers for Disease Control and Prevention. Scientific and technical information simply put. 2nd ed. Atlanta, GA: Office of Communication, Centers for Disease Control. 1999. 1999

Jackson RH, Davis TC, Bairnsfather LE, George RB, Crouch MA, Gault H. Patient reading ability: an overlooked problem in health care. South Med J.1991;84:1172-5.841172 1991 [CrossRef]

Streiff LD. Can clients understand our instructions? Image J Nurs Sch.1986;18:48-52.1848 1986 [CrossRef]

Meade CD, McKinney WP, Barnas GP. Educating patients with limited literacy skills: the effectiveness of printed and videotaped materials about colon cancer. Am J Public Health.1994;84:119-21.84119 1994 [CrossRef]

Murero M, D'Ancona G, Karamanoukian H. Use of the Internet by patients before and after cardiac surgery: telephone survey. J Med Internet Res.2001;3:E27.3E27 2001 [CrossRef]

Meade CD, Smith CF. Readability formulas: cautions and criteria. Patient Educ Couns.1991;17:153-8.17153 1991 [CrossRef]

McLaughlin GH. SMOG grading—a new readability formula. J Reading.1969;12:639-46.12639 1969

Clear and simple. Developing effective print materials for low-literate readers. Bethesda, MD: National Cancer Institute; 1994. Report no. NIH-95-3594. 1994

Hoffmann T, McKenna K. Analysis of stroke patients' and carers' reading ability and the content and design of written materials: recommendations for improving written stroke information. Patient Educ Couns.2006;60:286-93.60286 2006 [CrossRef]

Bass L. Health literacy: implications for teaching the adult patient. J Infus Nurs.2005;28:15-22.2815 2005 [CrossRef]

Wallace LS, Turner LW, Ballard JE, Keenum AJ, Weiss BD. Evaluation of web-based osteoporosis educational materials. J Womens Health (Larchmt).2005;14:936-45.14936 2005 [CrossRef]

Beredjiklian PK, Bozentka DJ, Steinberg DR, Bernstein J. Evaluating the source and content of orthopaedic information on the Internet. The case of carpal tunnel syndrome. J Bone Joint Surg Am.2000;82:1540-3.821540 2000

Purvis JM, Alexander AH, Einhorn TA, Griffin LY. American Orthopaedic Association symposium: Evaluating the flood of orthopaedic media and marketing information. J Bone Joint Surg Am.2005;87:1392-8.871392 2005 [CrossRef]

Hirsch SA. Academy introduces Web-based medical education for fellows and patients. J Bone Joint Surg Am.2000;82:1665-7.821665 2000

Topics

internet ; patient education ; pediatrics ; readability

Username

Password

Access for Patients

Forgot Password?

Have a subscription to the print edition?

Not a Subscriber?

Buy this Article

Get online access for 30 days for $35

New to JBJS?

Subscribe to the online edition for 30 days for $75

Register for a FREE limited account to get full access to all CME activities, to comment on public articles, or to sign up for alerts.

Have a subscription to the print edition?

Current subscribers to The Journal of Bone & Joint Surgery in either the print or quarterly DVD formats receive free online access to JBJS.org.

Activate your online account now

Forgot your password?

Enter your username and email address. We'll send you a reminder to the email address on record.

Username:*

Email Address:*

Forgot your username or need assistance? Please contact customer service at subs@jbjs.org. If your access is provided
by your institution, please contact you librarian or administrator for username and password information. Institutional
administrators, to reset your institution's master username or password, please contact subs@jbjs.org

References

Accreditation Statement

These activities have been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Academy of Orthopaedic Surgeons and The Journal of Bone and Joint Surgery, Inc. The American Academy of Orthopaedic Surgeons is accredited by the ACCME to provide continuing medical education for physicians.

CME Activities Associated with This Article

Submit a Comment

Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discretion of JBJS editorial staff.

* = Required Field

Comment Author(s) * (if multiple authors, separate names by comma)

Example: John Doe

Affiliation & Institution*

Comment Title*

Comment*

Cancel

Sanjeev Sabharwal, M.D.

Posted on March 14, 2008

Dr. Sabharwal et al. respond to Dr. Moseley, et al.

UMDNJ-New Jersey Medical School, Newark, NJ

To The Editor:

We agree with Dr. Moseley and coauthors that selecting the reading level for patient education materials is a â€œbalancing act with no perfect solutionâ€. Apparently, the target reading level for the AAOS patient education website, "Your Orthopaedic Connection" (YOC), was the eighth grade. However, we did report(1) that 70 percent of the online articles pertaining to pediatric orthopaedic topics had readability scores that were higher than the eighth grade level.

As mentioned in our manuscript, recommendations from the National Work Group on Cancer and Health(2), the American Medical Association(3), and the National Institutes of Health(4), are to keep readability of patient information material less than a sixth grade level. In another recent study(5) assessing the readability of all available patient education articles at the AAOS website, we found that only ten of the 426 articles in English had readability level less than or equal to sixth grade.

We concur with the authorsâ€™ suggestion of stratifying the contents of the patient education materials to suite the needs and comprehensibility of a larger segment of the population, and have mentioned this in the discussion section of our manuscript(1). We do realize that making a comprehensive patient education handout to serve the entire adult population is a challenging task and the current tools for assessing readability such as the Flesch-Kinkaid formula have limitations(1,5). However, we hope that organizations such as the AAOS and POSNA will continue to provide validated and up-to-date information about various orthopaedic conditions in a way that is sensitive to the diversity and readability of the entire patient population.

Referemces:

1. Badarudeen, S, and Sabharwal S. Readability of patient education materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America web sites. J Bone Joint Surg Am. 2008;90(1):199-204.

2. Cotugna N, Vickery CE, Carpenter-Haefele KM. Evaluation of literacy level of patient education pages in health-related journals. J Community Health. 2005;30(3):213-9.

3. Weiss BD. Health literacy: a manual for clinicians. Chicago, IL: American Medical Association Foundation. 2003.

4. National Institutes of Health: How to write easy to read health materials. Updated 2007 Aug 22. http://www.nlm.nih.gov.mlprox.csmc.edu/medlineplus/etr.html. Accessed 2007 Nov 8.

5. Sabharwal S, Badarudeen S, Kunju SU. Readability of online patient education materials at the AAOS website. Clin. Orthop. (Published online March 7, 2008).

Colin F. Moseley, M.D.

Posted on January 30, 2008

Readability of Online Educational Material

Editor, AAOS's "Your Orthopaedic Connection"

To The Editor:

We read with great interest the article, â€œReadability of Patient Education Materials from the American Academy of Orthopaedic and Pediatric Orthopaedic Society of North America Web Sites"(1). The authors examined readability levels of the pediatric articles on the American Academy of Orthopaedic Surgeons (AAOS) web site, "Your Orthopaedic Connection" (YOC), and the articles on the Pediatric Orthopaedic Society of North America (POSNA) web site, and found that only 2 percent of the articles were at the desired sixth grade readability level. That result is consistent with the fact that we have recently been targeting YOC articles at the eighth grade level.

The reading level of individuals seeking health information on the internet is not documented and, although the authors cite cancer literature to set their optimal readability at the sixth grade level, it has not been established that there is a single optimal readability level of health information on the internet. Indeed, lowering the reading level by restricting vocabulary and grammar can make it impossible to express essential concepts. Selecting the reading level is, therefore, a balancing act with no perfect solution.

To deal with the varied reading abilities and varied pre-existing "medical" knowledge of its users, YOC has decided to present information and learning modules at two levels: a "basic" level aimed at grade six English and little or no pre-existing knowledge, and an "advanced" level aimed at the high school graduate with strong reading skills and some knowledge of human biology. In a sense, the AAOS's "Orthopaedic Knowledge Online" presents a third, even higher, level aimed at the orthopaedic resident and surgeon.

The authors' suggestions to use shorter sentences and simpler words and phrases and to include more illustrations are established principles for future YOC content.

We applaud the authors for their novel, timely research on an important topic in this emerging field. We encourage their further investigations in this fertile area.

The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity. No commercial entity paid or directed, or agreed to pay or direct, any benefits to any research fund, foundation, division, center, clinical practice, or other charitable or nonprofit organization with which the authors, or a member of their immediate families, are affiliated or associated.

Reference:

1. Badarudeen S, Sabharwal S. Readability of patient education materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America web sites. J Bone Joint Surg Am. 2008;90:199-204.

Print
PDF
Email

Recipient(s) will receive an email with a link (good for 5 days) to 'Readability of Patient Education Materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America Web Sites'.
Your Name:*
Example: John Doe

Email Address:* CC Me:
Enter your valid email address. Example: jdoe@example.com

Recipient's Email Address:*

Separate multiple email address with semi-colons (up to 5).

Subject:* 's The Journal of Bone & Joint Surgery: 'Readability of Patient Education Materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America Web Sites'
Subject for your email.

Message:

(Optional, message will truncate at 1000 characters)

Processing your request... Please Wait...

Copyright © in the material you requested is held by The Journal of Bone & Joint Surgery, Inc. (unless otherwise noted). This email ability is provided as a courtesy, and by using it you agree that that you are requesting the material solely for personal, non-commercial use, and that it is subject to Journal of Bone & Joint Surgery, Inc.'s Terms of Use. The information provided in order to email this topic will not be used to send unsolicited email, nor will it be furnished to third parties. Please refer to The Journal of Bone & Joint Surgery Inc.'s Privacy Policy for further information.

Copyright © The Journal of Bone & Joint Surgery Inc. All rights reserved.
Share
Get Citation

Sameer Badarudeen, Sanjeev Sabharwal; Readability of Patient Education Materials from the American Academy of Orthopaedic Surgeons and Pediatric Orthopaedic Society of North America Web Sites. The Journal of Bone & Joint Surgery. 2008 Jan;90(1):199-204.

Download citation file:

RIS (Zotero)

EndNote

BibTex

Medlars

ProCite

RefWorks

Reference Manager

Copyright © The Journal of Bone and Joint Surgery, Inc.
Rights & Permissions
Article Alerts

Processing your request... Please Wait.
Submit a Comment
Data Supplements