Researchers and practitioners frequently use readability formulas to predict the suitability of health-related textsfor consumers (e.g., patient instructions, informed consent documents). However, the appropriateness of using readability formulas—originally developed for students and educational texts—for lay audiences and health-related texts remains to be validated. In this exploratory study, we compared two methods of assessing the readability of consumer health materials: the Cloze procedure, using actual readers, and readability formulas, using our Readability Analyzer program. A statistically significant inverse correlation (r = -0.581, p = 0.01) was found, suggesting that the Readability Analyzer may provide a reasonable “first approximation” for predicting readability of consumer health texts. We also identified several linguistic factors associated with increased reading ease as candidates for improving the performance of the Readability Analyzer. Our ultimate objective is to develop tools to support the design and evaluation of health information that is comprehensible and accessible to laypersons.