Many health writers say their materials are easy to read and suitable for readers of average skills, or below. When I ask how they determine this, writers often talk about measuring readability with grade-level formulas (also called “readability assessment tools” or “readability formulas”).
I cringe if this is the only way they assess readability, especially when writers rely solely on formulas within word-processing programs. Although arguably better than nothing, to me (and many others) such formulas are an imperfect and incomplete way to assess readability. Here are some reasons why:
Some of the most commonly used reading grade–level formulas (in-• cluding the Fry and the SMOG, discussed later in this chapter) were designed many years ago to help teachers select textbooks for stu- dents. These formulas were never intended to measure the readability of health materials written for adults. More than 100 factors impact how readable any given document is, • including its sentence length, word choice, organization, tone, lay- out, use of illustrations, and relevance to readers. But most reading grade–level formulas measure just a few of these factors. Commonly, they calculate readability based on the average number of words in sentences and the average number of syllables in words. Nonetheless, research shows that the length of sentences and length of words do
Osborne, M.Ed. Health Literacy From A to Z, Jones & Bartlett Learning, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/uneedu/detail.action?docID=3319392. Created from uneedu on 2018-03-21 13:14:06.
C op
yr ig
ht ©
2 01
1. J
on es
& B
ar tle
tt Le
ar ni
ng . A
ll rig
ht s
re se
rv ed
.
Assessing Readability with Grade Level Formulas16
impact reading difficulty the most, says Audrey Riffenbugh, MA, a specialist in plain language and readability and president of Plain Language Works, LLC. Studies have shown that among the general public, adults read, “on • the average, about five grade levels lower than the last year of school completed” (Doak, Doak, & Root 1996, p. 28). This means that some- one with a high school education would likely have difficulty reading materials written above a seventh-grade level. Readability formulas are just that—formulas. By looking at punctua-• tion (such as periods), a formula might determine that this statement is three sentences, not one: “Dr. Smith said I should call if my child has a temperature of 100.2.” And formulas do not take word order into account. In fact, these are assessed at the same reading level: “The order of words doesn’t matter at all” and “All at matter doesn’t words of order the.”
Despite these drawbacks, reading grade–level formulas are beneficial in that they are objective, easy to use, and the scores almost always get people’s attention. I often use such formulas at the beginning and end of plain-language projects to confirm that I made significant improvements. You can assess readability by hand or by computer.
StrategieS, ideaS, and SuggeStionS
Some helpful strategies are listed here:
Assess readability “by hand” (with pencil and paper). The SMOG (Simplified Measurement of Gobbledygook) and the Fry readability formula are two well-regarded reading grade–level tools that can be done by hand. While both give grade-level scores, they calculate them in different ways. The SMOG counts all words with three or more syllables in three 10-sentence passages. The Fry formula looks at the number of syllables and sentences in three 100-word passages. You can find detailed instructions for these and other readability tools on numerous Internet Web sites and writing guides.
Honestly, I like assessing readability by hand. I find the SMOG and Fry easy and quick to use; I can use these formulas with paper documents as well as computer files. I can also decide for myself whether a group of words is a complete sentence, a sentence fragment, or simply a title. And I espe- cially like seeing problem areas. For instance, with the SMOG you circle all multisyllabic words (words with three or more syllables) and so can quickly see which words to change.
Osborne, M.Ed. Health Literacy From A to Z, Jones & Bartlett Learning, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/uneedu/detail.action?docID=3319392. Created from uneedu on 2018-03-21 13:14:06.
C op
yr ig
ht ©
2 01
1. J
on es
& B
ar tle
tt Le
ar ni
ng . A
ll rig
ht s
re se
rv ed
.
Strategies, Ideas, and Suggestions 17
Example of Counting Multisyllabic Words by Using the SMOG
Directions for using the SMOG say to count all multisyllabic words (with three or more syllables, including those that are hyphenated) in a 10-sentence passage. Do this for three passages, and then total the number to find the reading grade level on a SMOG chart. Here is one of many places online to find a SMOG chart: http://www.sph .emory.edu/WELLNESS/reading.html.
I used the 10 sentences above, starting with “The SMOG . . . ” and ending with “. . . to change.” Here are the 30 multisyllabic words that I found:
Simplified, measurement, gobbledygook, readability, formula, well-regarded, grade-level, grade-level, calculate, different, syllables, 10-sentence, passages, formula, syllables, sentences, 100-word, passages, instructions, readability, numerous, Internet, assessing, readability, formulas, documents, com- puter, especially, multisyllabic, syllables.
Example of Counting Syllables for the Fry
The directions are to count all syllables in three 100-word passages. You also need to count the number of sentences in those 100 words, to the nearest tenth of a sentence. Do this three times, and then average the syllable scores as well as the sentence scores. You can then use these numbers to find the reading grade level on a Fry graph. Here is one of many online sources for the Fry graph: http://www .idph.state.ia.us/health_literacy/common/pdf/tools/fry.pdf.
Based on some sentences above, here is what syllable count starts to look like:
Hon-est-ly, I like ass-ess-ing read-a-bil-it-y by hand. I find the Fry and SMOG eas-y and quick to use. I can use these form-u-las with pa-per doc-u-ments as well as com-put-er fil-es. I can al-so de-cide for my-self wheth-er a group of words is a com-plete sen-tence, a sen-tence frag-ment, or simp-ly a ti-tle. And I es- pec-ial-ly like see-ing pro-blem a-re-as. For in-stance, with the SMOG you cir-cle . . .
Osborne, M.Ed. Health Literacy From A to Z, Jones & Bartlett Learning, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/uneedu/detail.action?docID=3319392. Created from uneedu on 2018-03-21 13:14:06.
C op
yr ig
ht ©
2 01
1. J
on es
& B
ar tle
tt Le
ar ni
ng . A
ll rig
ht s
re se
rv ed
.
Assessing Readability with Grade Level Formulas18
Stories from Practice: Prepare Text to Get Meaningful Results
I interviewed Audrey Riffenburgh of Plain Language Works, for an article about assessing readability. To get meaningful results with formulas, she reminds writers of guidelines given by the SMOG and Fry about how to prepare the text:
When possible, choose passages of text that are sufficiently • long, ideally with at least 30 sentences or 300 words. If the document you are evaluating is quite long, then assess at least three passages that represent the entire document—one each from the beginning, middle, and end. No matter the length of your document, avoid using the first • and last sentences, as these are often quite different from the reading level of the rest of the text.
Riffenburgh advocates also preparing text in these ways:
Remove headings, titles, subtitles, and bullet points. This way, • the computer does not give artificially low scores by counting headings and word fragments as complete sentences. Take out mid-sentence periods that might be part of an • abbreviation or number, such as “Dr.” or “100.2.” You need to do this because most com puter-based assessments are pro- grammed to recognize sentences as groups of words ending with a period, question mark, or exclamation. Be cautious not to overinterpret the results. Reading grade–• level scores are accurate only by plus or minus one and a half grade levels. This means that when a score drops from 7.3 to 6.8, the document is not necessarily easier to read. Results may vary depending on which formula you use. For example, the Fry formula often scores materials one to two grade levels lower than the SMOG.
Sources: Osborne (2000); Riffenburgh (2011).
Assess readability by computer. Many word processing programs include readability formulas within spelling/grammar check tools. Admittedly, these are fast and easy to use. But there are several limitations, including
Osborne, M.Ed. Health Literacy From A to Z, Jones & Bartlett Learning, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/uneedu/detail.action?docID=3319392. Created from uneedu on 2018-03-21 13:14:06.
C op
yr ig
ht ©
2 01
1. J
on es
& B
ar tle
tt Le
ar ni
ng . A
ll rig
ht s
re se
rv ed
.
Strategies, Ideas, and Suggestions 19
how these formulas count words and sentences. For instance, formulas may be programmed to count all dots as periods that end sentences (which is not always the case). Or have strict rules about syllables, such as determin- ing incorrectly that the word “people” has three syllables because of the two vowels next to each other.
Assess readability with specially designed software and online tools. My prefer- ence when assessing readability by computer is to use software designed for this purpose. In a recent search, I found a dazzling array of readability software and online assessment tools, tests, and calculators. Two readability programs that I like to use are:
Readability Calculations or Readability Plus, from Micro Power & • Light Co. (for PC or Macintosh), http://www.micropowerandlight .com/rd.html. Readability Studio (for PC), from OleanderSoftware, http://www • .oleandersolutions.com/readabilitystudio.html.
Follow, Do Not Fiddle with, Readability Rules
An area of dispute about readability formulas is whether it is okay to amend the rules and count repetitive, yet necessary, multisyllabic words (like “diabetes”) only once instead of each time they appear in the text. While I appreciate that fiddling with these formulas may lower the reading grade level, in my opinion this method can cause as many problems as it solves. For example, once you change this rule, you must create a new one about which multisyllabic words to count. Instead, my method is to follow all the formula’s rules but mention in a summary report why the grade-level score is as high as it is.
Explore other ways to assess readability. In addition to using reading grade– level formulas, I advocate using checklists and getting feedback from readers.
Checklists. • Checklists can be used throughout the writing process to objectively and subjectively assess a document’s strengths, weak- nesses, and areas to revise. You can rate content, design, organization,
Osborne, M.Ed. Health Literacy From A to Z, Jones & Bartlett Learning, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/uneedu/detail.action?docID=3319392. Created from uneedu on 2018-03-21 13:14:06.
C op
yr ig
ht ©
2 01
1. J
on es
& B
ar tle
tt Le
ar ni
ng . A
ll rig
ht s
re se
rv ed
.
Assessing Readability with Grade Level Formulas20
language, tone, appearance, graphics, cultural appropriateness, and other criteria using simple scales (such as “yes/no” or a rating from 1 to 3). One well-regarded checklist is the SAM (Suitability of Assessment of Materials) (Doak, Doak, and Root 1996). Testing materials with readers.• Most health literacy experts agree that the best way to know whether documents are truly easy to read is by asking for feedback from readers representing the intended audience. For example, if you are writing about issues affecting older adults, then you might ask for feedback from members of a local senior center. From your first idea to final draft, those representing your audience are the true experts about a document’s relevance, appeal, and readability. In my opinion, that’s the best assessment of all. (Read more in “Confirming Understanding: Feedback,” starting on page 35.)
“ Although arguably better than nothing, to me (and many others) reading grade-level formulas are an imperfect and incomplete way to assess readability. ”Ci t a t i o n s Doak C, Doak L, & Root J. 1996. Teaching Patients with Low Literacy Skills.
Philadelphia: J. B. Lippincott. Available at http://www.hsph.harvard.edu/ healthliteracy/resources/doak-book/. Accessed April 13, 2011.
Osborne H. 2000. In Other Words . . . Assessing Readability . . . Rules for Playing the Numbers Game. On Call. 3(12):38–39. Available at http://www.healthliteracy .com/assessing-readability. Accessed August 5, 2010.
Riffenburgh A. 2011. Personal communication.
so u r C e s t o Le a r n Mo r e
Armbruster BB, Osborn JH, Davison AI. 1985. Readability Formulas May Be Dangerous to Your Textbooks. Educational Leadership.18–20.
Chall JS. 1995. Readability Revisited: The New Dale-Chall Readability Formula. Cambridge, MA: Brookline Books/Lumen Editions.
European Commission. 2009, January 12. Guideline on the Readability of the Labelling and Package Leaflet of Medicinal Products for Human Use. European Medicines Agency, Revision 1. Available at http://ec.europa.eu/health/files/eudralex/vol-2/ c/2009_01_12_readability_guideline_final_en.pdf. Accessed April 13, 2011.
Gibson M, Hochhauser M. 2010. Readability Testing: European Performance vs. USA Formulas. SoCRA Source: A Publication of the Society of Clinical Research Associates. 64:69–71.
Osborne, M.Ed. Health Literacy From A to Z, Jones & Bartlett Learning, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/uneedu/detail.action?docID=3319392. Created from uneedu on 2018-03-21 13:14:06.
C op
yr ig
ht ©
2 01
1. J
on es
& B
ar tle
tt Le
ar ni
ng . A
ll rig
ht s
re se
rv ed
.
Sources to Learn More 21
Gilliam B, Pena SC, Mountain L. 1980. The Fry Graph Applied to Spanish Readability. Reading Teacher. 33(4):426–431.
Hochhauser M. 2008, February. Consent Form Readability and Grade Level Are Not the Same. SoCRA Source: A Publication of the Society of Clinical Research Associates. 55:35–36.
McLaughlin GH. 1974. Temptations of the Flesch. Instructional Science. 2(4):367–384. MedlinePlus. 2011. How to Write Easy-to-Read Health Materials. Available at http://
www.nlm.nih.gov/medlineplus/etr.html. Accessed August 5, 2010. Includes links to readability assessment tools as well as readability software programs.
Osborne H (host). 2010, June 29. Assessing Readability in the European Union (EU)
. Health Literacy Out Loud, no. 40. Available at http://www .healthliteracy.com/hlol-assessing-readability-in-eu. Accessed August 5, 2010.
Rugimbana R, Patel C. 1996. The Application of the Marketing Concept in Textbook Selection: Using the Cloze Procedure. Journal of Marketing Education. 18(1):14–20.
Stevens KC. 1980. Readability Formulae and McCall-Crabbs Standard Test Lessons in Reading. Reading Teacher. 33(4):413–415.
U.S. Department of Health & Human Services, Centers for Medicare & Medicaid Services, by Jeanne McGee of McGee & Evers Consulting. 2010. Toolkit Part 7: Using Readability Formulas: A Cautionary Note. Toolkit for Making Written Material Clear and Effective. Available at http://www.cms .gov/WrittenMaterialsToolkit/09_ToolkitPart07.asp#TopOfPage. Accessed January 3, 2011.
Osborne, M.Ed. Health Literacy From A to Z, Jones & Bartlett Learning, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/uneedu/detail.action?docID=3319392. Created from uneedu on 2018-03-21 13:14:06.
C op
yr ig
ht ©
2 01
1. J
on es
& B
ar tle
tt Le
ar ni
ng . A
ll rig
ht s
re se
rv ed
.
Osborne, M.Ed. Health Literacy From A to Z, Jones & Bartlett Learning, 2011. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/uneedu/detail.action?docID=3319392. Created from uneedu on 2018-03-21 13:14:06.
C op
yr ig
ht ©
2 01
1. J
on es
& B
ar tle
tt Le
ar ni
ng . A
ll rig
ht s
re se
rv ed