Evaluation of readability levels of online patient education materials for female pelvic floor disorders
Owing to the widespread accessibility of the Internet across many technological platforms, it is now recognized as the main source through which individuals seek health-related information. In the United States, over 80 proportion of adult Internet users utilize online search engines to obtain information pertaining to various health-related matters.  Although the information is easily attainable, its efficacy varies depending on its readability. Readability can be described as the characteristics of a text that make it easier to read than others.  Various readability tests are available on the internet. However, in modern linguistics, the Gunning Fox Index (GFI) and Flesch-Kinkaid (FK) readability tests are among the most frequently used instruments.  According to the recommendations set forth by the American Medical Association and the National Institutes of Health (NIH), patient educational materials are advised to maintain a reading level no higher than that of a sixth or 8 grade students . However, various studies have demonstrated that the literacy of online patient education materials (PEMs) is significantly above the recommended. [ – ] Female pelvic floor disorders, including pelvic organ prolapse, overactive bladder, urinary incontinence, and anal incontinence, were observed among the most searched health topics online 6 . Instead of seeking help, patients usually search websites for pelvic floor symptoms due to either embarrassment or misconceptions. Hence, obtaining accurate information about these taboo conditions is crucial for health promotion and education. Moreover, easy-to-understand information on pelvic floor disorders will not only guide women who need professional care, but will also prevent adverse events due to long-term self-coping strategies. Additionally, patients will be informed of the availability of a wide range of therapeutic options. As a result, patient awareness of their situation increases. Numerous PEMs for female pelvic floor disorders have had their readability tested and found to be more advanced than the American Medical Association-recommended 6th grade and NIH-recommended 8th grade education levels. [ – ] To the best of our knowledge, no readability analysis has been performed on Turkish PEMs for pelvic floor disorders in women. Turkish is the thirteenth most spoken language worldwide.  The Turkish language is distinguished by vowel harmony, extensive agglutination, and lack of grammatical gender. The extensive use of affixes can result in lengthy words and negatively impact readability scores. In this study, we aim to assess the readability of not only Turkish-translated PEMs from the web-sites of both the “International Urogynecological Association” (IUGA) and the “European Association of Urology” (EAU), but also the Turkish PEMS from the website of a national society, the “Society of Urological Surgery in Turkey” (SUS). 

A cross-sectional and observational study was performed using Turkish PEMs regarding pelvic floor disorders in women. All materials were downloaded on the same day from IUGA, EUA, and SUS publicly accessible websites in October 2021. On the IUGA patient education website ( https://www.yourpelvicfloor.org/ ), there were 47 PEMS, but 25 had Turkish language translations; all were included. The EAU designed a patient information project and launched a website ( https://patients.uroweb.org/tr/ ) to share educational materials. On this website, there were also materials on urological problems in male patients, but they were excluded; only 10 PEMs regarding urogynecology topics were included. The Society of Urological Surgery in Turkey shares a patient information booklet on their website ( https://urolojikcerrahidernegi.org/ ) about urological disorders, and 17 PEMs about urogynecological issues were extracted from this booklet. Figure shows the flow chart for the identification of PEMs evaluated in the study. The PEMs published by the IUGA and EUA were Turkish translations, whereas the PEMs published by the SUS were written directly in Turkish. Obstetrics and gynecology specialists translated the IUGA materials into Turkish. However, the source of the Turkish translation of the EAU PEMs has not been reported; translations were performed in collaboration with the Turkish Association of Urology and Society of Urological Surgery in Turkey. Societies provided no information on the linguistic validation of educational materials. Each item of the PEM was copied and pasted as ordinary text into Microsoft Word (Microsoft, Redmond, WA). All texts, including lists and titles, were included, whereas logos, emblems, and images that erroneously affected the calculations were omitted. The readability level was assessed using the Atesman formula, Flesch-Kincaid readability tests, and Gunning Fog index with the help of online readability tools.  The first 100 words for each PEM were analyzed. Since all data used in the study were available to the public, no ethical approval was necessary. 2.1. Atesman readability formula The Atesman readability formula was designed based on the analysis of word and sentence length. The formula for calculating the readability score is as follows: 198.825–40.175 × (total syllables/total words)  – 2.610 × (total words/total sentences).  According to the Atesman formula, the readability of a text is inversely proportional to its numerical value, with higher values indicating easier readability (near 100) and lower values indicating harder readability (closer to 0). Texts with a score below 50 are classified as difficult; texts with a score between 50 and 69 are classified as averagely difficult; and texts with a score of 70 and above are classified as easy and very easy (Table ). 2.2. Flesch-Kincaid Grade Level This assessment also considers the word and sentence length. The Flesch–Kincaid Grade Level formula represents a score in terms of an educational grade level in the United States. The formula for calculating readability is expressed as follows:0.39 × (total words/total sentences) + 11.8 × (total syllables/total words) – 15.59.  Based on the provided methodology, lower scores are indicative of a lower grade level and are associated with sections that are easy to read. Texts with a grade level equal to or >10 are considered difficult. Table presents the FK grade level and readability intervals. The text is classified as having a lower level of difficulty when the average number of syllables per word is closer to 1, whereas it is considered more challenging when the average number of syllables per word goes to ten. This principle is also applicable to the sentences. The text is classified as “simple” when the word count is reduced to 1, and as “challenging” when it exceeds ten. 2.3. Gunning Fog Index In contrast to the preceding 2 assessments, the Gunning Fog Index possesses 2 significant characteristics: the inclusion of words including 3 or more syllables and the calculation of the average sentence length in terms of word count. The Gunning Fog Index is derived from the formula 0.4 × [(words/sentences) + 100 × (complex words/words)].  Complex words are defined as words consisting of 3 or more syllables. Similar to FK grade level calculation, lower scores are associated with ease of readability in GFI and also describe the level of education needed to comprehend the calculated text. The text may be classified as simple if the final value falls in the range of 8 to 10, whereas it can be considered complex if the value exceeds 11 (see Table ). 2.4. Statistical analysis SPSS 28.0 (IBM Corp.; Armonk, NY) was used for statistical evaluation. One-way analysis of variance (ANOVA) was used to compare normally distributed variables and the Tukey test was used for post hoc analysis. The Kruskal–Wallis was used to compare non-normally distributed variables and the Dunn test was used for post hoc analysis. The level of significance for all comparisons was set at P < .05. 

The Atesman readability formula was designed based on the analysis of word and sentence length. The formula for calculating the readability score is as follows: 198.825–40.175 × (total syllables/total words)  – 2.610 × (total words/total sentences).  According to the Atesman formula, the readability of a text is inversely proportional to its numerical value, with higher values indicating easier readability (near 100) and lower values indicating harder readability (closer to 0). Texts with a score below 50 are classified as difficult; texts with a score between 50 and 69 are classified as averagely difficult; and texts with a score of 70 and above are classified as easy and very easy (Table ). 

This assessment also considers the word and sentence length. The Flesch–Kincaid Grade Level formula represents a score in terms of an educational grade level in the United States. The formula for calculating readability is expressed as follows:0.39 × (total words/total sentences) + 11.8 × (total syllables/total words) – 15.59.  Based on the provided methodology, lower scores are indicative of a lower grade level and are associated with sections that are easy to read. Texts with a grade level equal to or >10 are considered difficult. Table presents the FK grade level and readability intervals. The text is classified as having a lower level of difficulty when the average number of syllables per word is closer to 1, whereas it is considered more challenging when the average number of syllables per word goes to ten. This principle is also applicable to the sentences. The text is classified as “simple” when the word count is reduced to 1, and as “challenging” when it exceeds ten. 

In contrast to the preceding 2 assessments, the Gunning Fog Index possesses 2 significant characteristics: the inclusion of words including 3 or more syllables and the calculation of the average sentence length in terms of word count. The Gunning Fog Index is derived from the formula 0.4 × [(words/sentences) + 100 × (complex words/words)].  Complex words are defined as words consisting of 3 or more syllables. Similar to FK grade level calculation, lower scores are associated with ease of readability in GFI and also describe the level of education needed to comprehend the calculated text. The text may be classified as simple if the final value falls in the range of 8 to 10, whereas it can be considered complex if the value exceeds 11 (see Table ). 

SPSS 28.0 (IBM Corp.; Armonk, NY) was used for statistical evaluation. One-way analysis of variance (ANOVA) was used to compare normally distributed variables and the Tukey test was used for post hoc analysis. The Kruskal–Wallis was used to compare non-normally distributed variables and the Dunn test was used for post hoc analysis. The level of significance for all comparisons was set at P < .05. 

Among the 52 PEMs evaluated, 10 were from the EAU (Group 1), 25 from the IUGA (Group 2), and 17 from the SUS (Group 3). There was no statistically significant difference in the mean number of sentences, words, letters, or syllables in the PEMs. As shown in Table , the readability levels across PEMs sourced from different agencies were similar and on post hoc comparisons, no statistical significance was observed among the groups. Both Turkish-translated and originally written Turkish-language PEMs were at a higher readability level than the recommendations. When all Turkish PEMs were evaluated together for readability levels regardless of the publishing society, the median Atesman score was 64 (95% CI: 61.68–66.43), the median FK grade level was 16.8 (95% CI: 16.11–19.9) and the median GFI score was 19 (95 % CI: 18.2–19.9), respectively. The readability level of all PEMs was defined as at least averagely difficult according to the Atesman Formula and difficult according to the Gunning Fog Index. In addition, PEMs were very difficult according to the Flesch-Kincaid readability tests. 

The primary outcome of this study reveals that patient education resources related to female pelvic floor disorders in the Turkish language, available on the Internet, exhibit a level of complexity that exceeds the general population’s reading and comprehension abilities. The readability levels of all the PEMs examined did not meet the acceptable 8th grade standard set by the NIH.  This result is also consistent with the study of Haller and colleagues, who determined that PEMs formed by IUGA and the American Urogynecological Society had higher readability scores than recommended.  Betschart et al also found that PEMs provided by the EAU have higher readability levels.  Considering that the readability of these materials may exceed the comprehension level of ordinary patients, their potential influence on patient care may be constrained, particularly among groups with lower literacy rates. Individuals who possess limited comprehension abilities when it comes to healthcare educational materials are more susceptible to experiencing unfavorable health outcomes. These consequences include higher usage of emergency services, inadequate utilization of preventative healthcare services, improper use of prescription medications, and inadequate adherence to discharge instructions.  For example, misinformation regarding the use of a vaginal pessary can cause severe complications and even death due to sepsis in older patients. Serxner demonstrated that the implementation of an optimization strategy in patient education has positive outcomes, including enhanced satisfaction levels, increased confidence in medical decision-making, and improved adherence to healthcare programs.  To enhance the clarity and accessibility of our communication, we need to increase our knowledge of how to improve the readability of the written materials. To help clinicians prepare high-quality PEM, several organizations have made recommendations regarding the appropriate usage of plain language. [ – ] Enhancing the readability of PEMs has the potential to boost patient health literacy and eventually improve the patient-centered healthcare experience. There are several tools to assess readability, such as the Gunning Fog index, Flesch-Kincaid grade level formula, Simple Measure of Gobbledygook (SMOG) readability formula, and Fry graph readability formula, all of which were developed for written materials in English. The Atesman readability formula was adapted to Turkish from the Flesch-Kincaid grade level formula.  We used the Gunning Fog index, Flesch-Kincaid grade level formula, and Ateşman readability formula and found that the readability levels according to the different formulas were consistent. Notably, the NIH recommends that PEMs created in English should not be above the 8th grade level but does not provide any guidance on the appropriate readability level for PEMs written in languages other than English. To the best of our knowledge, no readability-level recommendations have yet been made for PEMs prepared in Turkish. The average length of education for women in Turkey is 8.5 years, according to a report by the Turkish Statistical Institute published in 2022. In the same report, it has been mentioned that 20.3% of women living in Turkey have more than a high school education.  In addition, Organisation for Economic Co-operation and Development has reported the mean literacy score for Turkish citizens as 227, which has been reported as 277 for people living in the United States of America.  Thus, to provide understandable and thereby usable health education materials in PFDs, the initial step should be the revision of the PEMs published on the websites of well-known international bodies according to the NIH-recommended readability level for English. To our knowledge, this is the first study to evaluate the readability level of Turkish PEMs regarding female pelvic floor disorders, but our study also has some limitations. Unfortunately, the ideal readability level for Turkish PEMs has not yet been described. Thus, we used NIH recommendations for readability assessment. However, as the average literacy level of the Turkish population is lower than that of the American population, the 8th grade readability level recommended by the NIH for health care related materials may not be appropriate for the Turkish population. The Atesman formula was created especially for Turkish; on the other hand, the FK test and the Gunning-Fog index were mainly used for English materials. However, several researchers have found similar readability levels for Turkish PEMs between Atesman and other readability formulas, as in our study.  Other instruments for readability are described in the literature. The SMOG is preferred for readability evaluation of health messages, but a strong correlation ( R = 0.95) between the Gunning-Fog index and SMOG has been previously reported.  Readability formulas are based on mathematical formulations according to sentence or word numbers but not on the intellectual content of the written material. We did not evaluate the scientific quality of the investigated PEMs; future studies can evaluate the scientific quality of Turkish PEMs with validated instruments, such as DISCERN. Increased awareness among women about the pelvic floor health problems that they can suffer during their lifetime is of great importance. Less than 50% of women with urinary incontinence seek help in their situation; however, after a comprehensive evaluation, most cases are treatable, but lack of knowledge is a barrier to accessing health care services.  At this point, national and international health societies provide information in English to patients but with complexity. The translation of complex materials into another language causes new complex materials to appear in different languages. To achieve readable translation, the original text also needs to have an appropriate reading level. International societies should pay utmost attention to readability rules when preparing materials for patient education. After educational materials are published, many national societies with mother languages other than English can share educational materials with their own populations through translation activities. However, national societies are also preparing materials in their mother languages with complexity, as shown in our study. Our results can be seen as a reflection of clinicians’ awareness of readability scores. Translated and native Turkish language materials were both above the recommended readability levels. The same situation can be true for other languages. Clinicians must control the complexity of the materials they provide to their patients. The management of pelvic floor diseases is generally accomplished using a personalized approach that considers the patient’s specific goals, prior experiences, and adherence to the treatment plan. Hence, it is important to provide patients with as easy-to-understand information as possible to maintain their active involvement in their healthcare decisions and plans. It is well known that readability and comprehension should be considered when commenting on the ease of the text. Avoiding the use of medical terminology is another issue that should be considered. Future work should focus on standardized measures and steps, considering both multidimensional comprehension and readability, to conduct user-friendly PEMs. 

The readability level of Turkish PEMs on the websites of national or international authorities is above the recommended levels. We need to simplify our sentences, optimize our vocabulary, and use simple patient-friendly sentences to explain our meaning. Improving the clarity of our PEMs will increase patient health literacy and ultimately improve the patient-centered experience in health care. On the other hand, healthcare authorities and international bodies should also take responsibility for providing recommendations for the preparation of PEMs, emphasizing the importance of readability scores and the use of plain languages with well-balanced content for the target population’s educational level. 

Conceptualization: Bulut Varli, Elif Nazli Cetindag, Kazibe Koyuncu Demir, Fulya Dokmeci. Data curation: Ulas Coban, Gunel Islamova. Formal analysis: Elif Nazli Cetindag, Ulas Coban, Gunel Islamova. Investigation: Bulut Varli, Elif Nazli Cetindag. Methodology: Elif Nazli Cetindag, Kazibe Koyuncu Demir. Supervision: Fulya Dokmeci. Writing – original draft: Bulut Varli, Elif Nazli Cetindag. Writing – review & editing: Bulut Varli, Fulya Dokmeci. 

