Psychometric Analysis of the eHealth Literacy Scale in Portuguese Older Adults (eHEALS-PT24): Instrument Development and Validation
Background Currently, the internet is the main source of health information. According to the survey Flash Eurobarometer 404 , 8 (80%) in 10 Europeans used the internet for private matters, with the majority (59%) using it for health information searching purposes. The increasing number of internet users has been influencing health systems globally, leading to investments in the provision of eHealth services. In recent years, there have been great efforts to use eHealth interventions to engage patients in health care and help them manage their health, particularly among older adults with chronic disease self-management and self-care . Nevertheless, despite the broad access to the internet and other electronic sources, there continues to be a lack of skills to access valid and reliable health information, as well as to understand, evaluate, and apply it adequately for decision-making purposes . Considering this condition, eHealth literacy has emerged as a new concept in health care and a challenge in public health . According to Norman and Skinner , eHealth literacy is “the ability to seek, find, understand, and evaluate information from electronic sources and apply it to knowledge gained to address or solve a health problem.” Based on this, individuals should be able to perform basic or advanced information retrieval, distinguish documents from reliable sources (eg, authoritative ones or scientific evidence-based), and understand eHealth terminology. Furthermore, this set of skills requires the ability to use information and communication technology (ICT), think critically about their nature, and efficiently navigate different electronic resources to obtain information for health-related decision-making. Considering that eHealth literacy is not a static set of skills, instead changing over time, Norman and Skinner  consider the importance of assessing and monitoring eHealth literacy as critical to supporting eHealth interventions. In recent years, indeed, older adults have become increasingly active on the internet and interested in digital health care services to manage their health; however, limited attention has been given to measuring eHealth literacy in this population. Therefore, we sought to address this gap by translating, culturally adapting, and validating a scale for Portuguese older adults that measure eHealth literacy levels . The eHealth Literacy Scale The eHealth Literacy Scale (eHEALS), developed by Norman and Skinner , was the most frequently used instrument to measure eHealth literacy worldwide, covering 18 languages, 26 countries, and diverse populations and contexts (ie, adolescents, adults, elderly, patients, healthy people, caregivers, health school professionals, community, and clinical practice) . Since its development, the eHEALS has been widely translated, adapted, and validated in several countries, such as Brazil , China , Ethiopia , Germany , Greece , Hungary , Indonesia , Iran , Italy , the Netherlands ; Norway , Poland , Portugal , Serbia , South Korea , Spain , and Sweden , as well as in countries with English as the main language . The scale’s development study  assessed the eHealth literacy of Canadian adolescents (n=664) aged between 13 and 21 years from 14 secondary schools. Considering the psychometrics test results, the instrument showed a good internal consistency (α=.88), with moderate to strong correlations between items (0.51≤ r ≤0.76) and a moderate test-retest reliability (0.40≤ r ≤0.68). In addition, a unidimensional structure was found in exploratory and confirmatory factor analyses, explaining 56% of the total variance. In Portugal, and similarly to the original study, the eHEALS was translated, adapted, and validated in 2014 in a sample of adolescents (n=1215) attending secondary education. The results showed a good internal consistency (α=.84) and a 2-factor structure in the exploratory factor analysis (EFA; extraction of 2 factors with α=.81 and α=.73), explaining 61% of the total variance. From a subgroup analysis, the authors found statistically significant differences regarding the level of education. For the variables sex and age, no statistically significant differences were found. No further translation, adaptation, and validation studies of eHEALS considering other target populations or contexts in Portugal have been carried out since then . Studies with samples including older adults  showed good internal consistency, with Cronbach α ranging between 0.87 and 0.99. Regarding factor analysis, 3 studies  obtained 1D structures, and 2 other studies obtained 3D ones . The questionnaire itself consists of 8 items assessing the person’s perception of knowledge, comfort, and ability to find, evaluate, and apply health ICT. Each of the items is scored at 5 points on a Likert scale ranging between “1” (totally disagree) and “5” (totally agree), with a total score from 8 to 40 points. The higher the score, the higher the eHealth literacy levels. Items 1 and 2 are related to awareness, items 3 and 4 are related to demand, items 6 and 7 are related to the evaluation of health resources, and items 5 and 8 are related to the use of health information. The instrument also contains 2 additional items, not adding to the final score, which assesses the participant’s perception of using the internet to access health information and make health decisions in terms of its usefulness and importance. The theoretical basis of the eHEALS was the social cognitive theory of Albert Bandura and the Lily Model, which explains multiple components of the constructs based on 6 components of literacies—traditional (literacy and numeracy skills), health, information, scientific, media, and computer . This paper presents and discusses the results of full psychometric testing of the eHEALS-PT24 (Portuguese version of the eHealth Literacy Scale for older adults), which aimed to: Translate, culturally adapt, and validate the eHEALS for assessing eHealth literacy in Portuguese older adults. Test full psychometric characteristics of the eHEALS to be used in Portuguese older adults. Explore associations between eHealth literacy and sociodemographic variables. 

Currently, the internet is the main source of health information. According to the survey Flash Eurobarometer 404 , 8 (80%) in 10 Europeans used the internet for private matters, with the majority (59%) using it for health information searching purposes. The increasing number of internet users has been influencing health systems globally, leading to investments in the provision of eHealth services. In recent years, there have been great efforts to use eHealth interventions to engage patients in health care and help them manage their health, particularly among older adults with chronic disease self-management and self-care . Nevertheless, despite the broad access to the internet and other electronic sources, there continues to be a lack of skills to access valid and reliable health information, as well as to understand, evaluate, and apply it adequately for decision-making purposes . Considering this condition, eHealth literacy has emerged as a new concept in health care and a challenge in public health . According to Norman and Skinner , eHealth literacy is “the ability to seek, find, understand, and evaluate information from electronic sources and apply it to knowledge gained to address or solve a health problem.” Based on this, individuals should be able to perform basic or advanced information retrieval, distinguish documents from reliable sources (eg, authoritative ones or scientific evidence-based), and understand eHealth terminology. Furthermore, this set of skills requires the ability to use information and communication technology (ICT), think critically about their nature, and efficiently navigate different electronic resources to obtain information for health-related decision-making. Considering that eHealth literacy is not a static set of skills, instead changing over time, Norman and Skinner  consider the importance of assessing and monitoring eHealth literacy as critical to supporting eHealth interventions. In recent years, indeed, older adults have become increasingly active on the internet and interested in digital health care services to manage their health; however, limited attention has been given to measuring eHealth literacy in this population. Therefore, we sought to address this gap by translating, culturally adapting, and validating a scale for Portuguese older adults that measure eHealth literacy levels . 

The eHealth Literacy Scale (eHEALS), developed by Norman and Skinner , was the most frequently used instrument to measure eHealth literacy worldwide, covering 18 languages, 26 countries, and diverse populations and contexts (ie, adolescents, adults, elderly, patients, healthy people, caregivers, health school professionals, community, and clinical practice) . Since its development, the eHEALS has been widely translated, adapted, and validated in several countries, such as Brazil , China , Ethiopia , Germany , Greece , Hungary , Indonesia , Iran , Italy , the Netherlands ; Norway , Poland , Portugal , Serbia , South Korea , Spain , and Sweden , as well as in countries with English as the main language . The scale’s development study  assessed the eHealth literacy of Canadian adolescents (n=664) aged between 13 and 21 years from 14 secondary schools. Considering the psychometrics test results, the instrument showed a good internal consistency (α=.88), with moderate to strong correlations between items (0.51≤ r ≤0.76) and a moderate test-retest reliability (0.40≤ r ≤0.68). In addition, a unidimensional structure was found in exploratory and confirmatory factor analyses, explaining 56% of the total variance. In Portugal, and similarly to the original study, the eHEALS was translated, adapted, and validated in 2014 in a sample of adolescents (n=1215) attending secondary education. The results showed a good internal consistency (α=.84) and a 2-factor structure in the exploratory factor analysis (EFA; extraction of 2 factors with α=.81 and α=.73), explaining 61% of the total variance. From a subgroup analysis, the authors found statistically significant differences regarding the level of education. For the variables sex and age, no statistically significant differences were found. No further translation, adaptation, and validation studies of eHEALS considering other target populations or contexts in Portugal have been carried out since then . Studies with samples including older adults  showed good internal consistency, with Cronbach α ranging between 0.87 and 0.99. Regarding factor analysis, 3 studies  obtained 1D structures, and 2 other studies obtained 3D ones . The questionnaire itself consists of 8 items assessing the person’s perception of knowledge, comfort, and ability to find, evaluate, and apply health ICT. Each of the items is scored at 5 points on a Likert scale ranging between “1” (totally disagree) and “5” (totally agree), with a total score from 8 to 40 points. The higher the score, the higher the eHealth literacy levels. Items 1 and 2 are related to awareness, items 3 and 4 are related to demand, items 6 and 7 are related to the evaluation of health resources, and items 5 and 8 are related to the use of health information. The instrument also contains 2 additional items, not adding to the final score, which assesses the participant’s perception of using the internet to access health information and make health decisions in terms of its usefulness and importance. The theoretical basis of the eHEALS was the social cognitive theory of Albert Bandura and the Lily Model, which explains multiple components of the constructs based on 6 components of literacies—traditional (literacy and numeracy skills), health, information, scientific, media, and computer . This paper presents and discusses the results of full psychometric testing of the eHEALS-PT24 (Portuguese version of the eHealth Literacy Scale for older adults), which aimed to: Translate, culturally adapt, and validate the eHEALS for assessing eHealth literacy in Portuguese older adults. Test full psychometric characteristics of the eHEALS to be used in Portuguese older adults. Explore associations between eHealth literacy and sociodemographic variables. 

Design We followed a methodology for translating, adapting, and validating instruments proposed by Sousa and Rojjanasrirat —a 7-step guideline based on the review of existing recommendations in the scientific literature on the process of translation, cultural adaptation, and validation of instruments for use in cross-cultural health care . The original version of the questionnaire was independently translated to Portuguese by 2 translators with high proficiency in English: a translator with knowledge of the terminology used in health care and another with knowledge of the cultural and linguistic characteristics of the target language (forward translation; step 1). After this, a third translator compared the two translated versions of the instrument and the original one to detect discrepancies. In addition, a first meeting with a committee of experts with different backgrounds and expertise was held to evaluate discrepancies, composed of the 2 bilingual experts from the first step, the third bilingual translator, academia members, and the research team (step 2). These 2 steps generated the preliminary translated version of the instrument for the Portuguese language. Independent, blinded backward translation to English was carried out by 2 translators whose first language was English as spoken in the United Kingdom: one translator with knowledge of the terminology used in health care and another with knowledge of the cultural and linguistic characteristics of the target language (step 3). This step generated 2 retroverted versions of the original version of the instrument, which were then compared with the original one (in English) to detect discrepancies by the research team. In addition, a second meeting with a committee of experts with different backgrounds and expertise was held to detect discrepancies, composed of all translators involved in the previous three steps, academia members, and the research team (step 4). These 2 steps generated the prefinal version of the translated instrument, which the original authors validated. The prefinal version of the instrument was then tested in a pilot study in the target language with a monolingual sample (n=15) to evaluate the instructions, items, and response format clarity (ie, to establish whether the instrument could be satisfactorily understood and completed by people from the target population; step 5). As part of this step, a committee of experts (n=8) was also held to further examine the instrument for clarity of the instructions, items, and response format (content equivalence assessment). The fifth step generated some adjustments in the prefinal version of the instrument. According to the protocol followed, step 6 (preliminary psychometric testing of the prefinal version of the translated instrument with a bilingual sample) is rarely used, except when a bilingual population is accessible, which justified the option of not carrying it out in this validation process. Full psychometric testing (step 7) involved reviewing and refining the items of the final version of the eHEALS for Portuguese older adults, as well as establishing stability, internal consistency, validity, and model fit. This step encompassed 2 substudies: general psychometric characteristics of the instrument (n=80, study 1) and confirmatory factor analysis (CFA; n=301, study 2). The sample sizes were defined according to the protocol, namely, at least 10 participants per item of the instrument for general psychometric analysis and 300 to 500 participants for CFA. Participants and Recruitment Study 1 included 80 older adults from a Health Family Unit (Primary Care) located in Faro. The sampling procedures were based on a convenience sample. Data collection was conducted by in-person questionnaires between May and July 2022. During that time, patients aged 65 years or older who went to the Health Family Unit (Primary Care) for an appointment were invited to participate in the survey. Study 2 included 301 older adults from 2 distinct Health Family Units (Primary Care), located in Faro and Lisbon. Participants were randomly selected from the total number of patients of the 2 considered Health Family Units and invited by phone to visit the unit and participate in the survey. Data collection was conducted by in-person questionnaires between May and July 2023. Inclusion criteria were age (aged 65 years or older) and sex (female and male). Difficulty communicating or using the Portuguese language was an exclusion criterion. The questionnaire removal was performed when the questionnaires were not entirely fulfilled. Data Analysis The data analysis was conducted in several stages to ensure a comprehensive understanding of the psychometric properties of the eHEALS-PT24. The following steps outline our analytical approach. Descriptive Statistics Initially, we performed descriptive statistical analyses to understand the basic features of the data. This included calculating mean, SD, frequencies, and percentages for all the variables, providing a snapshot of the sample characteristics. Scale and Items Stability To assess the scale and item stability, we calculated Pearson correlations. We considered the following Pearson correlation ranges: very weak (0.00-0.19), weak (0.20-0.39), moderate (0.40-0.59), strong (0.60 and 0.79), and very strong (0.80-0.99). Also, statistical significance was set at P <.001 . Reliability Analysis To assess the internal consistency, we calculated Cronbach α coefficients. A Cronbach α value of 0.70 or higher was considered indicative of acceptable reliability. The item-total correlation between the items and the total score was considered high, with values above 0.40 . Construct Validity: Factor Analysis We conducted an EFA to explore the underlying factor structure of the scale. The Kaiser-Meyer-Olkin (KMO) measure and Bartlett test of sphericity were used to assess the suitability of the data for factor analysis. Factors were extracted using principal component analysis, and a varimax rotation was applied to aid interpretability. Following EFA, a CFA was performed using structural equation modeling to verify the factor structure obtained from EFA. Model fit was evaluated using different indices, including the comparative fit index (CFI), Tucker-Lewis Index (TLI), root mean square error of approximation (RMSEA), and chi-square ( df ) ratio. RMSEA values of 0.00 to 0.05 indicate a closer or good fit, 0.05 to 0.08 a fair fit, 0.08 to 0.10 a mediocre fit, and over 0.10 a poor fit. Other fit indexes that we used were the CFI and TLI, both of which should be close to 0.95 . Construct Validity: Known-Groups Validity The known-groups validity measures an instrument’s ability to distinguish among distinct groups (ie, discriminating groups known to differ on the variable of interest) . To evaluate the known-groups validity, we compared eHEALS-PT24 scores across different sociodemographic groups (ie, age, sex, residence area, and education level) using the independent Student t test (2-tailed), ANOVA, or Kruskal-Wallis test, as appropriate. Statistical Software All analyses were performed using IBM SPSS and Jamovi (The Jamovi Project). By using this multistep analytical process, we aimed to rigorously evaluate the psychometric properties of the eHEALS-PT24, ensuring its reliability and validity in assessing eHealth literacy among Portuguese older adults. Ethical Considerations The ethics review assessments were submitted and approved by Portugal’s National Data Protection Authority and Regional Health Administrations (Algarve, Lisbon, and Tagus Valley; (approval 4825/2018). Informed consent was applied to participants, according to the model recommended by the Ethics Committee of the Algarve Regional Health Administration, guaranteeing voluntary participation, the possibility of withdrawing at any time, data anonymity, privacy, confidentiality, and no harm to participants. In addition, an authorization request was made to the original authors of the eHEALS , which was accepted. 

We followed a methodology for translating, adapting, and validating instruments proposed by Sousa and Rojjanasrirat —a 7-step guideline based on the review of existing recommendations in the scientific literature on the process of translation, cultural adaptation, and validation of instruments for use in cross-cultural health care . The original version of the questionnaire was independently translated to Portuguese by 2 translators with high proficiency in English: a translator with knowledge of the terminology used in health care and another with knowledge of the cultural and linguistic characteristics of the target language (forward translation; step 1). After this, a third translator compared the two translated versions of the instrument and the original one to detect discrepancies. In addition, a first meeting with a committee of experts with different backgrounds and expertise was held to evaluate discrepancies, composed of the 2 bilingual experts from the first step, the third bilingual translator, academia members, and the research team (step 2). These 2 steps generated the preliminary translated version of the instrument for the Portuguese language. Independent, blinded backward translation to English was carried out by 2 translators whose first language was English as spoken in the United Kingdom: one translator with knowledge of the terminology used in health care and another with knowledge of the cultural and linguistic characteristics of the target language (step 3). This step generated 2 retroverted versions of the original version of the instrument, which were then compared with the original one (in English) to detect discrepancies by the research team. In addition, a second meeting with a committee of experts with different backgrounds and expertise was held to detect discrepancies, composed of all translators involved in the previous three steps, academia members, and the research team (step 4). These 2 steps generated the prefinal version of the translated instrument, which the original authors validated. The prefinal version of the instrument was then tested in a pilot study in the target language with a monolingual sample (n=15) to evaluate the instructions, items, and response format clarity (ie, to establish whether the instrument could be satisfactorily understood and completed by people from the target population; step 5). As part of this step, a committee of experts (n=8) was also held to further examine the instrument for clarity of the instructions, items, and response format (content equivalence assessment). The fifth step generated some adjustments in the prefinal version of the instrument. According to the protocol followed, step 6 (preliminary psychometric testing of the prefinal version of the translated instrument with a bilingual sample) is rarely used, except when a bilingual population is accessible, which justified the option of not carrying it out in this validation process. Full psychometric testing (step 7) involved reviewing and refining the items of the final version of the eHEALS for Portuguese older adults, as well as establishing stability, internal consistency, validity, and model fit. This step encompassed 2 substudies: general psychometric characteristics of the instrument (n=80, study 1) and confirmatory factor analysis (CFA; n=301, study 2). The sample sizes were defined according to the protocol, namely, at least 10 participants per item of the instrument for general psychometric analysis and 300 to 500 participants for CFA. 

Study 1 included 80 older adults from a Health Family Unit (Primary Care) located in Faro. The sampling procedures were based on a convenience sample. Data collection was conducted by in-person questionnaires between May and July 2022. During that time, patients aged 65 years or older who went to the Health Family Unit (Primary Care) for an appointment were invited to participate in the survey. Study 2 included 301 older adults from 2 distinct Health Family Units (Primary Care), located in Faro and Lisbon. Participants were randomly selected from the total number of patients of the 2 considered Health Family Units and invited by phone to visit the unit and participate in the survey. Data collection was conducted by in-person questionnaires between May and July 2023. Inclusion criteria were age (aged 65 years or older) and sex (female and male). Difficulty communicating or using the Portuguese language was an exclusion criterion. The questionnaire removal was performed when the questionnaires were not entirely fulfilled. 

The data analysis was conducted in several stages to ensure a comprehensive understanding of the psychometric properties of the eHEALS-PT24. The following steps outline our analytical approach. Descriptive Statistics Initially, we performed descriptive statistical analyses to understand the basic features of the data. This included calculating mean, SD, frequencies, and percentages for all the variables, providing a snapshot of the sample characteristics. Scale and Items Stability To assess the scale and item stability, we calculated Pearson correlations. We considered the following Pearson correlation ranges: very weak (0.00-0.19), weak (0.20-0.39), moderate (0.40-0.59), strong (0.60 and 0.79), and very strong (0.80-0.99). Also, statistical significance was set at P <.001 . Reliability Analysis To assess the internal consistency, we calculated Cronbach α coefficients. A Cronbach α value of 0.70 or higher was considered indicative of acceptable reliability. The item-total correlation between the items and the total score was considered high, with values above 0.40 . Construct Validity: Factor Analysis We conducted an EFA to explore the underlying factor structure of the scale. The Kaiser-Meyer-Olkin (KMO) measure and Bartlett test of sphericity were used to assess the suitability of the data for factor analysis. Factors were extracted using principal component analysis, and a varimax rotation was applied to aid interpretability. Following EFA, a CFA was performed using structural equation modeling to verify the factor structure obtained from EFA. Model fit was evaluated using different indices, including the comparative fit index (CFI), Tucker-Lewis Index (TLI), root mean square error of approximation (RMSEA), and chi-square ( df ) ratio. RMSEA values of 0.00 to 0.05 indicate a closer or good fit, 0.05 to 0.08 a fair fit, 0.08 to 0.10 a mediocre fit, and over 0.10 a poor fit. Other fit indexes that we used were the CFI and TLI, both of which should be close to 0.95 . Construct Validity: Known-Groups Validity The known-groups validity measures an instrument’s ability to distinguish among distinct groups (ie, discriminating groups known to differ on the variable of interest) . To evaluate the known-groups validity, we compared eHEALS-PT24 scores across different sociodemographic groups (ie, age, sex, residence area, and education level) using the independent Student t test (2-tailed), ANOVA, or Kruskal-Wallis test, as appropriate. Statistical Software All analyses were performed using IBM SPSS and Jamovi (The Jamovi Project). By using this multistep analytical process, we aimed to rigorously evaluate the psychometric properties of the eHEALS-PT24, ensuring its reliability and validity in assessing eHealth literacy among Portuguese older adults. 

Initially, we performed descriptive statistical analyses to understand the basic features of the data. This included calculating mean, SD, frequencies, and percentages for all the variables, providing a snapshot of the sample characteristics. 

To assess the scale and item stability, we calculated Pearson correlations. We considered the following Pearson correlation ranges: very weak (0.00-0.19), weak (0.20-0.39), moderate (0.40-0.59), strong (0.60 and 0.79), and very strong (0.80-0.99). Also, statistical significance was set at P <.001 . 

To assess the internal consistency, we calculated Cronbach α coefficients. A Cronbach α value of 0.70 or higher was considered indicative of acceptable reliability. The item-total correlation between the items and the total score was considered high, with values above 0.40 . 

We conducted an EFA to explore the underlying factor structure of the scale. The Kaiser-Meyer-Olkin (KMO) measure and Bartlett test of sphericity were used to assess the suitability of the data for factor analysis. Factors were extracted using principal component analysis, and a varimax rotation was applied to aid interpretability. Following EFA, a CFA was performed using structural equation modeling to verify the factor structure obtained from EFA. Model fit was evaluated using different indices, including the comparative fit index (CFI), Tucker-Lewis Index (TLI), root mean square error of approximation (RMSEA), and chi-square ( df ) ratio. RMSEA values of 0.00 to 0.05 indicate a closer or good fit, 0.05 to 0.08 a fair fit, 0.08 to 0.10 a mediocre fit, and over 0.10 a poor fit. Other fit indexes that we used were the CFI and TLI, both of which should be close to 0.95 . 

The known-groups validity measures an instrument’s ability to distinguish among distinct groups (ie, discriminating groups known to differ on the variable of interest) . To evaluate the known-groups validity, we compared eHEALS-PT24 scores across different sociodemographic groups (ie, age, sex, residence area, and education level) using the independent Student t test (2-tailed), ANOVA, or Kruskal-Wallis test, as appropriate. 

All analyses were performed using IBM SPSS and Jamovi (The Jamovi Project). By using this multistep analytical process, we aimed to rigorously evaluate the psychometric properties of the eHEALS-PT24, ensuring its reliability and validity in assessing eHealth literacy among Portuguese older adults. 

The ethics review assessments were submitted and approved by Portugal’s National Data Protection Authority and Regional Health Administrations (Algarve, Lisbon, and Tagus Valley; (approval 4825/2018). Informed consent was applied to participants, according to the model recommended by the Ethics Committee of the Algarve Regional Health Administration, guaranteeing voluntary participation, the possibility of withdrawing at any time, data anonymity, privacy, confidentiality, and no harm to participants. In addition, an authorization request was made to the original authors of the eHEALS , which was accepted. 

Pilot Study Before the full psychometric testing, we carried out a pilot study, which we briefly summarized to support the results presented in this subsection. The sampling procedure for piloting was based on snowball sampling, composed of older people aged 65 years or older (n=15). Data collection was conducted using a web-based questionnaire between July and August 2020. The interrater agreement among the sample was above 80% for all items. As part of this step, a committee of experts (n=8) was also held to further examine the instrument for clarity of the instructions, items, and response format; also, to assess the content equivalence, content validity index at the item level (I-CVI=1) and at the scale level (S-CVI=1), as well as the Fleiss κ coefficient of agreement (κ=0.24; P <.001), were calculated. Participant’s Characteristics Study 1 Out of the 80 participants, the sample comprised 47 male (58.8%) and 33 female (41.3%) participants, with a mean age of 71 (SD 5.26) years, ranging from 65 to 88 years. From the total sample (n=80), most of the participants were married or had civil partners (n=63, 78.8%), living with their spouse (n=65, 81.3%) in urban areas (n=70, 87.5%). In addition, most of the respondents were retired (n=69, 86.3%). Regarding formal education, this was analyzed according to the International Standard Classification of Education (ISCED), showing that the most general education level was upper secondary education—ISCED 3 (n=20, 25%), followed by primary education—ISCED 1 (n=17, 21.3%), and bachelor or equivalent degree—ISCED 6 (n=17, 21.3%). Concerning the current health status, most participants reported a diagnosed chronic condition (n=58, 72.5%), mainly high blood pressure (n=29, 36.3%), diabetes (n=25, 31.2%), and dyslipidemia (n=8, 10%). Regarding prescribed medication, 85% (n=68) of the sample answered that they usually take medicines, with 3 being the average number of medicines per person and 9 being the maximum number. presents the sociodemographic data. Study 2 The sample comprised 170 male (56.5%) and 131 female (43.5%) participants, with a mean age of 72 (SD 5.15) years, ranging from 65 to 88 years. From the total sample (n=301), most of the participants were married or had civil partners (n=215, 71.4%), living with their spouse (n=267, 88.7%) in urban areas (n=267, 88.7%). In addition, most of the respondents were retired (n=271, 90%). Regarding formal education, this was analyzed according to the ISCED, showing that the most general education level was primary education—ISCED 1 (n=116, 38.5%), followed by lower secondary education—ISCED 2 (n=51, 17%) and bachelor or equivalent degree—ISCED 6 (n=37, 12.3%). Concerning the current health status, most participants reported a diagnosed chronic condition (n=252, 83.7%), mainly diabetes (n=179, 59.5%), dyslipidemia (n=45, 15%), and high blood pressure (n=116, 38.5%). Regarding prescribed medication, 92.7% (n=279) of the sample answered that they usually take medicines, with 3 being the average number of medicines per person and 10 being the maximum number. In this study, 19 questionnaires were excluded since they were not entirely fulfilled. presents the sociodemographic data of both studies. Stability Study 1 From the stability analysis (Pearson correlation coefficient) of the eHEALS-PT24, correlations between all items of the scale were identified, ranging between moderate and strong (except for one very strong correlation), with statistical significance ( P <.001). The lowest correlation value ( r =0.42) was between items 1 and 8 (moderate correlation). In turn, the highest value ( r =0.91) was a very strong correlation between items 2 and 3 . Study 2 From the stability analysis (Pearson correlation coefficient) of the eHEALS-PT24, correlations between all items of the scale were identified, ranging between moderate and strong (except for one very strong correlation), with statistical significance ( P <.001). The lowest correlation value ( r =0.81) was between items 1 and 8 (strong correlation). In turn, the highest value ( r =0.96) was a very strong correlation between items 2 and 3 . Reliability Study 1 The analysis of the internal consistency of the eHEALS-PT24 showed an adequate Cronbach α coefficient (Cronbach α=0.92). The statistics after excluding 1 of the 8 items did not indicate an increase in reliability: the value of Cronbach α ranged from 0.90 to 0.92. The mean total value in the eHealth literacy for the sample (n=80) was 27.25 (SD 5.61). The average score for each item was 3, ranging between 3.09 (item 8) and 3.55 (item 1). Concerning item-total correlation, coefficients above 0.4 for all items showed that the items were consistent with each other and correlated with the final score . Study 2 The analysis of the internal consistency of the eHEALS-PT24 showed an adequate Cronbach α coefficient (Cronbach α=0.98). The statistics after excluding 1 of the 8 items did not indicate an increase in reliability, with Cronbach α coefficient values remaining stable at 0.98. The mean total value in the eHealth literacy for the sample (n=301) was 23.31 (SD 9.53). The average score for each item was 3, ranging between 2.7 (item 8) and 2.99 (items 1 and 4). Concerning item-total correlation, coefficients above 0.40 for all items showed that the items were consistent with each other and correlated with the final score . Construct Validity: EFA The Bartlett sphericity test corroborated the factorability of the correlation matrix for both studies (study 1: χ 2 28 =446.87, P <.001; study 2: χ 2 28 =3932.81, P <.001). In addition, the KMO test value demonstrated adequate sampling for both studies (study 1: KMO=0.87; study 2: KMO=0.94). Given the quality of Bartlett and KMO values, the criteria for factor analysis were gathered. By performing the EFA, we considered factors with eigenvalues above 1 for testing the structure of the instrument. In accordance with the original structure of the eHEALS , the Jamovi software extracted 1 factor for the structure of the eHEALS-PT24 for both studies (Study 1 and 2). Study 1 In the EFA of study 1 (n=80), a single factor showed moderate to strong loadings (0.57 to 0.89; ) but poor model fit indicators: RMSEA was high (0.20), TLI was below the threshold (0.79), and there was a significant chi-square test ( χ 2 20 =83.7, P <.001). The factor explained 58.3% of the total variance. Study 2 In the EFA of study 2 (n=301), a single factor showed moderate to strong loadings (0.89 to 0.95) but poor model fit indicators: RMSEA was high (0.20), TLI was marginally acceptable (0.91), and there was a significant chi-square test ( χ 2 20 =271, P <.001). The factor explained 86.4% of the total variance. The single-factor structure of eHEALS-PT24 for both studies (studies 1 and 2) was also empirically confirmed on screen plots ( and ). Construct Validity: CFA Regarding CFA for study 2 (n=301), a single-factor model demonstrated high item loadings (standardized estimates 0.88 to 0.97; ); however, the model fit was mixed: the chi-square test was significant ( χ 2 20 =265, P <.001), CFI was acceptable (0.94), while TLI (0.91) was marginally acceptable, and RMSEA (0.20) indicated a poor fit. Given the RMSEA values for one dimension, and due to the variability of factor structures in the literature, we compared the indexes for 1-, 2-, and 3-factor structures in study 2 . For the 2-factor model, we followed Tomás et al , dividing items 1-5 (factor 1) and items 6-8 (factor 2). High item loadings (standardized estimates 0.90 to 0.97) were obtained ; however, the model fit was mixed: CFI was acceptable (0.96), TLI (0.94) was marginally acceptable, RMSEA (0.17) indicated a poor fit, and the chi-square test was significant ( χ 2 19 =188, P <.001; ). For the 3-factor model, we followed Stellefson et al , dividing items 1-2 (factor 1), items 3-5 (factor 2), and items 6-8 (factor 3). High item loadings (standardized estimates 0.90 to 0.98) were obtained ; however, the model fit was mixed: CFI was acceptable (0.96), TLI (0.93) was marginally acceptable, RMSEA (0.18) indicated a poor fit, and the chi-square test was significant ( χ 2 17 =179, P <.001; ). Interpretation of the eHEALS-PT24 Scores We used the mean total and SD values to determine high or low eHealth literacy levels. Therefore, participants were divided into 2 groups for each study as follows: Study 1: (1) low eHealth literacy levels (≤27 points) and (2) high eHealth literacy levels (>32 points). Study 2: (1) low eHealth literacy levels (≤23 points) and (2) high eHealth literacy levels (>31 points). In study 1 (n=80), 35 (43.8%) participants had low eHealth literacy levels, and 9 (11.3%) had high levels. In study 2 (n=301), 115 (38.2%) participants had low eHealth literacy levels, and 71 (23.6%) had high levels. Construct Validity: Known-Groups We performed known-groups validity in study 2, as presented in the following subsections. Association Between eHealth Literacy and Age (Student t test and Pearson Correlation) To analyze the association between eHealth literacy and age, participants were distributed into two different groups: (1) 65-79 years and (2) ≥80 years. The first group comprised 269 (89.4%) participants, and the second group comprised 32 (10.6%) participants. The analysis showed differences statistically significant between eHealth literacy levels (total scale value) and age (t 299 =3.94; P <.001). Individuals aged 65-79 years had, on average, 7.0 points higher eHEALS scores than those aged ≥80 years. We also carried out a correlation analysis between both variables (Pearson correlation), which showed that variables are inversely related, which means the greater one variable, the smaller the other ( r =–0.355, P <.001). Association Between eHealth Literacy and Sex (Student t test) Female respondents showed an average of 23.68 (SD 9.89) points and male respondents showed an average of 23.03 (SD 9.26) points; however, no statistically significant differences between eHealth literacy levels (total scale value) and sex were found (t 299 =0.59; P =.31). Association Between eHealth Literacy and Residence Area (Student t test) Statistically significant differences between eHealth literacy levels (total scale value) and residence area were found (t 299 =4.19; P <.001). Participants living in urban areas showed an average of 24.11 (SD 9.16) points, and those living in rural areas showed an average of 17.03 (SD 10.15) points. Association Between eHealth Literacy and Education Level (1-Way ANOVA or Kruskal-Wallis Test) Statistically significant differences between eHealth literacy levels (total scale value) and education level were found ( F 8 =10.24; P <.001), with the highest mean values corresponding to the education levels of doctoral or equivalent level (ISCED 8; mean score of eHealth literacy levels: 31.57, SD 4.20), master’s degree or equivalent (ISCED 7; mean 29.17, SD 5.60), and postsecondary but not tertiary education (ISCED 4; mean 27.68, SD 8.52). Since some of the education levels had nonrepresentative samples (ie, ISCEDs 2, 4, 5, 7, and 8), we tested the association between eHealth literacy and education level using Kruskal-Wallis test (H=58.56; P <.001). In this analysis, we found statistically significant differences between participants with primary education (ISCED 1) and all other education levels, with an exception for lower secondary education (ISCED 2) and master’s degree or equivalent (ISCED 7). No other statistically significant differences were found. The mean (SD) and significance of eHEALS-PT24 by subgroups (ie, age, sex, residence area, and education level) calculated in study 2 are presented in . 

Before the full psychometric testing, we carried out a pilot study, which we briefly summarized to support the results presented in this subsection. The sampling procedure for piloting was based on snowball sampling, composed of older people aged 65 years or older (n=15). Data collection was conducted using a web-based questionnaire between July and August 2020. The interrater agreement among the sample was above 80% for all items. As part of this step, a committee of experts (n=8) was also held to further examine the instrument for clarity of the instructions, items, and response format; also, to assess the content equivalence, content validity index at the item level (I-CVI=1) and at the scale level (S-CVI=1), as well as the Fleiss κ coefficient of agreement (κ=0.24; P <.001), were calculated. 

Study 1 Out of the 80 participants, the sample comprised 47 male (58.8%) and 33 female (41.3%) participants, with a mean age of 71 (SD 5.26) years, ranging from 65 to 88 years. From the total sample (n=80), most of the participants were married or had civil partners (n=63, 78.8%), living with their spouse (n=65, 81.3%) in urban areas (n=70, 87.5%). In addition, most of the respondents were retired (n=69, 86.3%). Regarding formal education, this was analyzed according to the International Standard Classification of Education (ISCED), showing that the most general education level was upper secondary education—ISCED 3 (n=20, 25%), followed by primary education—ISCED 1 (n=17, 21.3%), and bachelor or equivalent degree—ISCED 6 (n=17, 21.3%). Concerning the current health status, most participants reported a diagnosed chronic condition (n=58, 72.5%), mainly high blood pressure (n=29, 36.3%), diabetes (n=25, 31.2%), and dyslipidemia (n=8, 10%). Regarding prescribed medication, 85% (n=68) of the sample answered that they usually take medicines, with 3 being the average number of medicines per person and 9 being the maximum number. presents the sociodemographic data. Study 2 The sample comprised 170 male (56.5%) and 131 female (43.5%) participants, with a mean age of 72 (SD 5.15) years, ranging from 65 to 88 years. From the total sample (n=301), most of the participants were married or had civil partners (n=215, 71.4%), living with their spouse (n=267, 88.7%) in urban areas (n=267, 88.7%). In addition, most of the respondents were retired (n=271, 90%). Regarding formal education, this was analyzed according to the ISCED, showing that the most general education level was primary education—ISCED 1 (n=116, 38.5%), followed by lower secondary education—ISCED 2 (n=51, 17%) and bachelor or equivalent degree—ISCED 6 (n=37, 12.3%). Concerning the current health status, most participants reported a diagnosed chronic condition (n=252, 83.7%), mainly diabetes (n=179, 59.5%), dyslipidemia (n=45, 15%), and high blood pressure (n=116, 38.5%). Regarding prescribed medication, 92.7% (n=279) of the sample answered that they usually take medicines, with 3 being the average number of medicines per person and 10 being the maximum number. In this study, 19 questionnaires were excluded since they were not entirely fulfilled. presents the sociodemographic data of both studies. 

Out of the 80 participants, the sample comprised 47 male (58.8%) and 33 female (41.3%) participants, with a mean age of 71 (SD 5.26) years, ranging from 65 to 88 years. From the total sample (n=80), most of the participants were married or had civil partners (n=63, 78.8%), living with their spouse (n=65, 81.3%) in urban areas (n=70, 87.5%). In addition, most of the respondents were retired (n=69, 86.3%). Regarding formal education, this was analyzed according to the International Standard Classification of Education (ISCED), showing that the most general education level was upper secondary education—ISCED 3 (n=20, 25%), followed by primary education—ISCED 1 (n=17, 21.3%), and bachelor or equivalent degree—ISCED 6 (n=17, 21.3%). Concerning the current health status, most participants reported a diagnosed chronic condition (n=58, 72.5%), mainly high blood pressure (n=29, 36.3%), diabetes (n=25, 31.2%), and dyslipidemia (n=8, 10%). Regarding prescribed medication, 85% (n=68) of the sample answered that they usually take medicines, with 3 being the average number of medicines per person and 9 being the maximum number. presents the sociodemographic data. 

The sample comprised 170 male (56.5%) and 131 female (43.5%) participants, with a mean age of 72 (SD 5.15) years, ranging from 65 to 88 years. From the total sample (n=301), most of the participants were married or had civil partners (n=215, 71.4%), living with their spouse (n=267, 88.7%) in urban areas (n=267, 88.7%). In addition, most of the respondents were retired (n=271, 90%). Regarding formal education, this was analyzed according to the ISCED, showing that the most general education level was primary education—ISCED 1 (n=116, 38.5%), followed by lower secondary education—ISCED 2 (n=51, 17%) and bachelor or equivalent degree—ISCED 6 (n=37, 12.3%). Concerning the current health status, most participants reported a diagnosed chronic condition (n=252, 83.7%), mainly diabetes (n=179, 59.5%), dyslipidemia (n=45, 15%), and high blood pressure (n=116, 38.5%). Regarding prescribed medication, 92.7% (n=279) of the sample answered that they usually take medicines, with 3 being the average number of medicines per person and 10 being the maximum number. In this study, 19 questionnaires were excluded since they were not entirely fulfilled. presents the sociodemographic data of both studies. 

Study 1 From the stability analysis (Pearson correlation coefficient) of the eHEALS-PT24, correlations between all items of the scale were identified, ranging between moderate and strong (except for one very strong correlation), with statistical significance ( P <.001). The lowest correlation value ( r =0.42) was between items 1 and 8 (moderate correlation). In turn, the highest value ( r =0.91) was a very strong correlation between items 2 and 3 . Study 2 From the stability analysis (Pearson correlation coefficient) of the eHEALS-PT24, correlations between all items of the scale were identified, ranging between moderate and strong (except for one very strong correlation), with statistical significance ( P <.001). The lowest correlation value ( r =0.81) was between items 1 and 8 (strong correlation). In turn, the highest value ( r =0.96) was a very strong correlation between items 2 and 3 . 

From the stability analysis (Pearson correlation coefficient) of the eHEALS-PT24, correlations between all items of the scale were identified, ranging between moderate and strong (except for one very strong correlation), with statistical significance ( P <.001). The lowest correlation value ( r =0.42) was between items 1 and 8 (moderate correlation). In turn, the highest value ( r =0.91) was a very strong correlation between items 2 and 3 . 

From the stability analysis (Pearson correlation coefficient) of the eHEALS-PT24, correlations between all items of the scale were identified, ranging between moderate and strong (except for one very strong correlation), with statistical significance ( P <.001). The lowest correlation value ( r =0.81) was between items 1 and 8 (strong correlation). In turn, the highest value ( r =0.96) was a very strong correlation between items 2 and 3 . 

Study 1 The analysis of the internal consistency of the eHEALS-PT24 showed an adequate Cronbach α coefficient (Cronbach α=0.92). The statistics after excluding 1 of the 8 items did not indicate an increase in reliability: the value of Cronbach α ranged from 0.90 to 0.92. The mean total value in the eHealth literacy for the sample (n=80) was 27.25 (SD 5.61). The average score for each item was 3, ranging between 3.09 (item 8) and 3.55 (item 1). Concerning item-total correlation, coefficients above 0.4 for all items showed that the items were consistent with each other and correlated with the final score . Study 2 The analysis of the internal consistency of the eHEALS-PT24 showed an adequate Cronbach α coefficient (Cronbach α=0.98). The statistics after excluding 1 of the 8 items did not indicate an increase in reliability, with Cronbach α coefficient values remaining stable at 0.98. The mean total value in the eHealth literacy for the sample (n=301) was 23.31 (SD 9.53). The average score for each item was 3, ranging between 2.7 (item 8) and 2.99 (items 1 and 4). Concerning item-total correlation, coefficients above 0.40 for all items showed that the items were consistent with each other and correlated with the final score . 

The analysis of the internal consistency of the eHEALS-PT24 showed an adequate Cronbach α coefficient (Cronbach α=0.92). The statistics after excluding 1 of the 8 items did not indicate an increase in reliability: the value of Cronbach α ranged from 0.90 to 0.92. The mean total value in the eHealth literacy for the sample (n=80) was 27.25 (SD 5.61). The average score for each item was 3, ranging between 3.09 (item 8) and 3.55 (item 1). Concerning item-total correlation, coefficients above 0.4 for all items showed that the items were consistent with each other and correlated with the final score . 

The analysis of the internal consistency of the eHEALS-PT24 showed an adequate Cronbach α coefficient (Cronbach α=0.98). The statistics after excluding 1 of the 8 items did not indicate an increase in reliability, with Cronbach α coefficient values remaining stable at 0.98. The mean total value in the eHealth literacy for the sample (n=301) was 23.31 (SD 9.53). The average score for each item was 3, ranging between 2.7 (item 8) and 2.99 (items 1 and 4). Concerning item-total correlation, coefficients above 0.40 for all items showed that the items were consistent with each other and correlated with the final score . 

The Bartlett sphericity test corroborated the factorability of the correlation matrix for both studies (study 1: χ 2 28 =446.87, P <.001; study 2: χ 2 28 =3932.81, P <.001). In addition, the KMO test value demonstrated adequate sampling for both studies (study 1: KMO=0.87; study 2: KMO=0.94). Given the quality of Bartlett and KMO values, the criteria for factor analysis were gathered. By performing the EFA, we considered factors with eigenvalues above 1 for testing the structure of the instrument. In accordance with the original structure of the eHEALS , the Jamovi software extracted 1 factor for the structure of the eHEALS-PT24 for both studies (Study 1 and 2). Study 1 In the EFA of study 1 (n=80), a single factor showed moderate to strong loadings (0.57 to 0.89; ) but poor model fit indicators: RMSEA was high (0.20), TLI was below the threshold (0.79), and there was a significant chi-square test ( χ 2 20 =83.7, P <.001). The factor explained 58.3% of the total variance. Study 2 In the EFA of study 2 (n=301), a single factor showed moderate to strong loadings (0.89 to 0.95) but poor model fit indicators: RMSEA was high (0.20), TLI was marginally acceptable (0.91), and there was a significant chi-square test ( χ 2 20 =271, P <.001). The factor explained 86.4% of the total variance. The single-factor structure of eHEALS-PT24 for both studies (studies 1 and 2) was also empirically confirmed on screen plots ( and ). 

In the EFA of study 1 (n=80), a single factor showed moderate to strong loadings (0.57 to 0.89; ) but poor model fit indicators: RMSEA was high (0.20), TLI was below the threshold (0.79), and there was a significant chi-square test ( χ 2 20 =83.7, P <.001). The factor explained 58.3% of the total variance. 

In the EFA of study 2 (n=301), a single factor showed moderate to strong loadings (0.89 to 0.95) but poor model fit indicators: RMSEA was high (0.20), TLI was marginally acceptable (0.91), and there was a significant chi-square test ( χ 2 20 =271, P <.001). The factor explained 86.4% of the total variance. The single-factor structure of eHEALS-PT24 for both studies (studies 1 and 2) was also empirically confirmed on screen plots ( and ). 

Regarding CFA for study 2 (n=301), a single-factor model demonstrated high item loadings (standardized estimates 0.88 to 0.97; ); however, the model fit was mixed: the chi-square test was significant ( χ 2 20 =265, P <.001), CFI was acceptable (0.94), while TLI (0.91) was marginally acceptable, and RMSEA (0.20) indicated a poor fit. Given the RMSEA values for one dimension, and due to the variability of factor structures in the literature, we compared the indexes for 1-, 2-, and 3-factor structures in study 2 . For the 2-factor model, we followed Tomás et al , dividing items 1-5 (factor 1) and items 6-8 (factor 2). High item loadings (standardized estimates 0.90 to 0.97) were obtained ; however, the model fit was mixed: CFI was acceptable (0.96), TLI (0.94) was marginally acceptable, RMSEA (0.17) indicated a poor fit, and the chi-square test was significant ( χ 2 19 =188, P <.001; ). For the 3-factor model, we followed Stellefson et al , dividing items 1-2 (factor 1), items 3-5 (factor 2), and items 6-8 (factor 3). High item loadings (standardized estimates 0.90 to 0.98) were obtained ; however, the model fit was mixed: CFI was acceptable (0.96), TLI (0.93) was marginally acceptable, RMSEA (0.18) indicated a poor fit, and the chi-square test was significant ( χ 2 17 =179, P <.001; ). 

We used the mean total and SD values to determine high or low eHealth literacy levels. Therefore, participants were divided into 2 groups for each study as follows: Study 1: (1) low eHealth literacy levels (≤27 points) and (2) high eHealth literacy levels (>32 points). Study 2: (1) low eHealth literacy levels (≤23 points) and (2) high eHealth literacy levels (>31 points). In study 1 (n=80), 35 (43.8%) participants had low eHealth literacy levels, and 9 (11.3%) had high levels. In study 2 (n=301), 115 (38.2%) participants had low eHealth literacy levels, and 71 (23.6%) had high levels. 

We performed known-groups validity in study 2, as presented in the following subsections. Association Between eHealth Literacy and Age (Student t test and Pearson Correlation) To analyze the association between eHealth literacy and age, participants were distributed into two different groups: (1) 65-79 years and (2) ≥80 years. The first group comprised 269 (89.4%) participants, and the second group comprised 32 (10.6%) participants. The analysis showed differences statistically significant between eHealth literacy levels (total scale value) and age (t 299 =3.94; P <.001). Individuals aged 65-79 years had, on average, 7.0 points higher eHEALS scores than those aged ≥80 years. We also carried out a correlation analysis between both variables (Pearson correlation), which showed that variables are inversely related, which means the greater one variable, the smaller the other ( r =–0.355, P <.001). Association Between eHealth Literacy and Sex (Student t test) Female respondents showed an average of 23.68 (SD 9.89) points and male respondents showed an average of 23.03 (SD 9.26) points; however, no statistically significant differences between eHealth literacy levels (total scale value) and sex were found (t 299 =0.59; P =.31). Association Between eHealth Literacy and Residence Area (Student t test) Statistically significant differences between eHealth literacy levels (total scale value) and residence area were found (t 299 =4.19; P <.001). Participants living in urban areas showed an average of 24.11 (SD 9.16) points, and those living in rural areas showed an average of 17.03 (SD 10.15) points. Association Between eHealth Literacy and Education Level (1-Way ANOVA or Kruskal-Wallis Test) Statistically significant differences between eHealth literacy levels (total scale value) and education level were found ( F 8 =10.24; P <.001), with the highest mean values corresponding to the education levels of doctoral or equivalent level (ISCED 8; mean score of eHealth literacy levels: 31.57, SD 4.20), master’s degree or equivalent (ISCED 7; mean 29.17, SD 5.60), and postsecondary but not tertiary education (ISCED 4; mean 27.68, SD 8.52). Since some of the education levels had nonrepresentative samples (ie, ISCEDs 2, 4, 5, 7, and 8), we tested the association between eHealth literacy and education level using Kruskal-Wallis test (H=58.56; P <.001). In this analysis, we found statistically significant differences between participants with primary education (ISCED 1) and all other education levels, with an exception for lower secondary education (ISCED 2) and master’s degree or equivalent (ISCED 7). No other statistically significant differences were found. The mean (SD) and significance of eHEALS-PT24 by subgroups (ie, age, sex, residence area, and education level) calculated in study 2 are presented in . 

To analyze the association between eHealth literacy and age, participants were distributed into two different groups: (1) 65-79 years and (2) ≥80 years. The first group comprised 269 (89.4%) participants, and the second group comprised 32 (10.6%) participants. The analysis showed differences statistically significant between eHealth literacy levels (total scale value) and age (t 299 =3.94; P <.001). Individuals aged 65-79 years had, on average, 7.0 points higher eHEALS scores than those aged ≥80 years. We also carried out a correlation analysis between both variables (Pearson correlation), which showed that variables are inversely related, which means the greater one variable, the smaller the other ( r =–0.355, P <.001). 

Female respondents showed an average of 23.68 (SD 9.89) points and male respondents showed an average of 23.03 (SD 9.26) points; however, no statistically significant differences between eHealth literacy levels (total scale value) and sex were found (t 299 =0.59; P =.31). 

Statistically significant differences between eHealth literacy levels (total scale value) and residence area were found (t 299 =4.19; P <.001). Participants living in urban areas showed an average of 24.11 (SD 9.16) points, and those living in rural areas showed an average of 17.03 (SD 10.15) points. 

Statistically significant differences between eHealth literacy levels (total scale value) and education level were found ( F 8 =10.24; P <.001), with the highest mean values corresponding to the education levels of doctoral or equivalent level (ISCED 8; mean score of eHealth literacy levels: 31.57, SD 4.20), master’s degree or equivalent (ISCED 7; mean 29.17, SD 5.60), and postsecondary but not tertiary education (ISCED 4; mean 27.68, SD 8.52). Since some of the education levels had nonrepresentative samples (ie, ISCEDs 2, 4, 5, 7, and 8), we tested the association between eHealth literacy and education level using Kruskal-Wallis test (H=58.56; P <.001). In this analysis, we found statistically significant differences between participants with primary education (ISCED 1) and all other education levels, with an exception for lower secondary education (ISCED 2) and master’s degree or equivalent (ISCED 7). No other statistically significant differences were found. The mean (SD) and significance of eHEALS-PT24 by subgroups (ie, age, sex, residence area, and education level) calculated in study 2 are presented in . 

Principal Findings and Comparison With Previous Work In this paper, we have presented the translation, cultural adaptation, and validation of the eHEALS for Portuguese older adults (eHEALS-PT24) by carrying out a general psychometric analysis in a sample of 80 participants (study 1) and a CFA including 301 participants (study 2). For this discussion, we highlight the findings obtained in study 2 since it represents the final stage of the validation of eHEALS-PT24, according to the followed protocol . Based on scale and items stability and reliability analysis, we concluded that the eHEALS-PT24 is a reliable tool for measuring eHealth literacy. Correlations between all items of the eHEALS-PT24, ranging between moderate and strong ones, with statistical significance, indicated that the questions were related to the instrument with no redundancy. In addition, the instrument showed good internal consistency. All items were considered reliable to assess eHealth literacy levels in older adults, with high correlations between each other. Compared with the original validation study , internal consistency was higher (Cronbach α=0.88 vs 0.98), which is comparable with previous studies among older adults . The only study showing a higher Cronbach α than ours was a validation in older Hispanic participants  (Cronbach α=0.99), but the sample was small (n=20), which was pointed out as a limitation by the authors. Concerning construct validity, the eHEALS was originally developed with a single-factor structure , and in our study, the same structure was yielded in EFA, explaining 86.4% of the total variance. This total variance explained was higher compared with the original validation study  (56%), as well as to previous studies among older adults , suggesting the model captured a significant portion of the data’s variation. Regarding CFA, a single-factor model demonstrated high item loadings; however, the model fit was mixed: the chi-square test was significant ( P <.001), CFI (0.94) and TLI (0.91) were both acceptable, and RMSEA (0.20) indicated a poor fit. The finding of a unidimensional structure also aligns with previous studies carried out in older adults ; however, it contradicts the results of the 2 other studies conducted among older adults, which indicated a better fit for the 3-factor structure . The variability of factor structures in the literature and the RMSEA value obtained led us to compare the indices (ie, chi-square, CFI, TLI, and RMSEA) for 1-, 2-, and 3-factor structures. All indexes fitted well, but RMSEA remained poor (ie, 0.20 for 1 factor, 0.17 for 2 factors, and 0.18 for 3 factors), suggesting that the model may not fit the data well. These findings are in line with the eHEALS Korean validation study (CFI=0.95; TLI=0.92; and RMSEA=0.12) , which followed a unidimensional structure. Considering the original validation study , as well as others who used Rasch analysis (or other analyses under modern test theory) , we also decided on the 1-factor structure. The eHEALS-PT24 mean score was 23.31, with 38.2% (115/301) of participants showing low eHealth literacy levels and 23.6% (71/301) showing high levels. Compared with other validation studies among older adults, the mean score was higher than in Spain (22.35)  but lower than in China (30.94)  and in the United States (29.05) . As shown by previous studies , item 8 of the eHEALS-PT24 had the lowest average among all scale items, indicating that people do not feel so confident in using information from the internet to make health-related decisions compared with other eHealth literacy skills. To further examine construct validity, known-groups validity was assessed, showing statistically significant differences in demographic variables (age, residence area, and education levels), except for sex. While a previous study among older adults  found no differences between eHealth literacy levels and age, we concluded that the greater the age, the smaller the eHealth literacy. Those aged 65-79 years had, on average, 7.0 points higher (eHEALS scores) than those aged ≥80 years. Regarding eHealth literacy levels and sex, Aponte and Nokes  pointed out significant differences, where mean values of eHEALS scores for men (13.85) were significantly lower than those in women (25.77). In our study, female respondents had, on average, 0.65 points higher eHEALS scores than male respondents; however, no statistically significant differences were found between eHealth literacy levels and sex ( P =.31). We also tested the association of eHealth literacy levels and the residence area, finding statistically significant differences ( P <.001), where participants living in urban areas had on average 7.08 points higher (eHEALS scores) than those living in rural areas. Regarding education levels, the highest mean values corresponded to doctoral or equivalent level (ISCED 8; 31.57), master’s degree or equivalent (ISCED 7; 29.17), and postsecondary but not tertiary education (ISCED 4; 27.68). Since some education-level cases had nonrepresentative samples (n<30), this result should be read with caution. Therefore, we can only infer participants with primary education (ISCED 1) had significant differences in eHealth literacy levels from the participants with other education levels, except for lower secondary education (ISCED 2) and master’s degree or equivalent (ISCED 7). Strengths and Limitations There are some limitations in this study. First, there was limited geographic representation since the sample did not include older people from all regions of the country (only Faro and Lisbon). If we consider, however, the sample representativeness, we believe it is suitable for providing us with an adequate snapshot of our population of interest. Second, since the web-based questionnaires were only applied in the pilot study, we did not have any rigorous confirmatory process to determine that participants were aged, in fact, aged 65 years or older. Third, similarly to other validation studies , test-retest reliability was not conducted. As such, further research could address reliability and responsiveness over time. Fourth, although a more comprehensive psychometric analysis could be conducted to establish the instrument validity, such as investigating correlations with external measures of eHealth literacy or health outcomes, the results of this study are promising and show that the eHEALS-PT24 is a reliable tool for perceived measurement of eHealth literacy. Fifth, the eHEALS instrument measures a narrow scope of eHealth literacy and may not fully capture the complex concept of eHealth literacy nowadays since it was developed before the social media era (Web 2.0). Nevertheless, the benefits of eHEALS (strong psychometric properties, brevity, and usability) outweigh its shortcomings, and the eHEALS arguably serves as a convenient instrument for health professionals. Conclusions In conclusion, the eHEALS-PT24 is a reliable and valid instrument for assessing and monitoring the eHealth literacy of Portuguese older adults. This instrument can be useful in identifying older adults who are able to use eHealth resources and participate in eHealth interventions aimed at engaging them in health care and helping them to manage their health and, by extension, assess the effects of eHealth interventions. 

In this paper, we have presented the translation, cultural adaptation, and validation of the eHEALS for Portuguese older adults (eHEALS-PT24) by carrying out a general psychometric analysis in a sample of 80 participants (study 1) and a CFA including 301 participants (study 2). For this discussion, we highlight the findings obtained in study 2 since it represents the final stage of the validation of eHEALS-PT24, according to the followed protocol . Based on scale and items stability and reliability analysis, we concluded that the eHEALS-PT24 is a reliable tool for measuring eHealth literacy. Correlations between all items of the eHEALS-PT24, ranging between moderate and strong ones, with statistical significance, indicated that the questions were related to the instrument with no redundancy. In addition, the instrument showed good internal consistency. All items were considered reliable to assess eHealth literacy levels in older adults, with high correlations between each other. Compared with the original validation study , internal consistency was higher (Cronbach α=0.88 vs 0.98), which is comparable with previous studies among older adults . The only study showing a higher Cronbach α than ours was a validation in older Hispanic participants  (Cronbach α=0.99), but the sample was small (n=20), which was pointed out as a limitation by the authors. Concerning construct validity, the eHEALS was originally developed with a single-factor structure , and in our study, the same structure was yielded in EFA, explaining 86.4% of the total variance. This total variance explained was higher compared with the original validation study  (56%), as well as to previous studies among older adults , suggesting the model captured a significant portion of the data’s variation. Regarding CFA, a single-factor model demonstrated high item loadings; however, the model fit was mixed: the chi-square test was significant ( P <.001), CFI (0.94) and TLI (0.91) were both acceptable, and RMSEA (0.20) indicated a poor fit. The finding of a unidimensional structure also aligns with previous studies carried out in older adults ; however, it contradicts the results of the 2 other studies conducted among older adults, which indicated a better fit for the 3-factor structure . The variability of factor structures in the literature and the RMSEA value obtained led us to compare the indices (ie, chi-square, CFI, TLI, and RMSEA) for 1-, 2-, and 3-factor structures. All indexes fitted well, but RMSEA remained poor (ie, 0.20 for 1 factor, 0.17 for 2 factors, and 0.18 for 3 factors), suggesting that the model may not fit the data well. These findings are in line with the eHEALS Korean validation study (CFI=0.95; TLI=0.92; and RMSEA=0.12) , which followed a unidimensional structure. Considering the original validation study , as well as others who used Rasch analysis (or other analyses under modern test theory) , we also decided on the 1-factor structure. The eHEALS-PT24 mean score was 23.31, with 38.2% (115/301) of participants showing low eHealth literacy levels and 23.6% (71/301) showing high levels. Compared with other validation studies among older adults, the mean score was higher than in Spain (22.35)  but lower than in China (30.94)  and in the United States (29.05) . As shown by previous studies , item 8 of the eHEALS-PT24 had the lowest average among all scale items, indicating that people do not feel so confident in using information from the internet to make health-related decisions compared with other eHealth literacy skills. To further examine construct validity, known-groups validity was assessed, showing statistically significant differences in demographic variables (age, residence area, and education levels), except for sex. While a previous study among older adults  found no differences between eHealth literacy levels and age, we concluded that the greater the age, the smaller the eHealth literacy. Those aged 65-79 years had, on average, 7.0 points higher (eHEALS scores) than those aged ≥80 years. Regarding eHealth literacy levels and sex, Aponte and Nokes  pointed out significant differences, where mean values of eHEALS scores for men (13.85) were significantly lower than those in women (25.77). In our study, female respondents had, on average, 0.65 points higher eHEALS scores than male respondents; however, no statistically significant differences were found between eHealth literacy levels and sex ( P =.31). We also tested the association of eHealth literacy levels and the residence area, finding statistically significant differences ( P <.001), where participants living in urban areas had on average 7.08 points higher (eHEALS scores) than those living in rural areas. Regarding education levels, the highest mean values corresponded to doctoral or equivalent level (ISCED 8; 31.57), master’s degree or equivalent (ISCED 7; 29.17), and postsecondary but not tertiary education (ISCED 4; 27.68). Since some education-level cases had nonrepresentative samples (n<30), this result should be read with caution. Therefore, we can only infer participants with primary education (ISCED 1) had significant differences in eHealth literacy levels from the participants with other education levels, except for lower secondary education (ISCED 2) and master’s degree or equivalent (ISCED 7). 

There are some limitations in this study. First, there was limited geographic representation since the sample did not include older people from all regions of the country (only Faro and Lisbon). If we consider, however, the sample representativeness, we believe it is suitable for providing us with an adequate snapshot of our population of interest. Second, since the web-based questionnaires were only applied in the pilot study, we did not have any rigorous confirmatory process to determine that participants were aged, in fact, aged 65 years or older. Third, similarly to other validation studies , test-retest reliability was not conducted. As such, further research could address reliability and responsiveness over time. Fourth, although a more comprehensive psychometric analysis could be conducted to establish the instrument validity, such as investigating correlations with external measures of eHealth literacy or health outcomes, the results of this study are promising and show that the eHEALS-PT24 is a reliable tool for perceived measurement of eHealth literacy. Fifth, the eHEALS instrument measures a narrow scope of eHealth literacy and may not fully capture the complex concept of eHealth literacy nowadays since it was developed before the social media era (Web 2.0). Nevertheless, the benefits of eHEALS (strong psychometric properties, brevity, and usability) outweigh its shortcomings, and the eHEALS arguably serves as a convenient instrument for health professionals. 

In conclusion, the eHEALS-PT24 is a reliable and valid instrument for assessing and monitoring the eHealth literacy of Portuguese older adults. This instrument can be useful in identifying older adults who are able to use eHealth resources and participate in eHealth interventions aimed at engaging them in health care and helping them to manage their health and, by extension, assess the effects of eHealth interventions. 

