Evaluating the psychometric properties of the eHealth Literacy Scale in Brazilian adults

Rev Bras Enferm. 2022;75(1): e20201320 https://doi.org/10.1590/0034-7167-2020-1320 8 of ABSTRACT Objective: To translate and adapt the eHealth Literacy Scale for the cultural reality of Brazil and to evaluate the psychometric properties of its Brazilian Portuguese version. Methods: The instrument was translated and adapted to Brazilian Portuguese and applied to a sample of 502 individuals from 18 to 80 years old who lived in the surrounding areas of six Family Health Units of a city in the countryside of the state of São Paulo, Brazil. The data was evaluated using exploratory and confirmatory factor analysis, item response theory, and instrument reliability measures (Cronbach’s alpha and McDonald’s omega). Results: The eHealth Literacy Scale Brazilian version (eHEALS-Br) presented an excellent internal consistency (α = 0.95 e ω = 0.95), with only one dimension and an explained variation of 81.79%. Conclusions: The Brazilian version of the instrument showed excellent psychometric properties to measure the levels of digital health literacy in adults from the country. Descriptors: Health Literacy; Validation Studies; Computer Literacy; Unified Health System; Primary Health Care.


INTRODUCTION
Health literacy is seen by the World Health Organization (WHO) as an important social health determinant (1) . It is a construct which considers people's knowledge, confidence, and skills to access, understand, judge, and apply the information in their process of decision making in health (1) .
The transmission of health information through the Internet has been a worldwide trend, making it the main media for health communication (2) , and there is a wide variety of information available online (3)(4) . In the Brazilian context, the percentage of residences that use the Internet increased, between 2016 and 2017, from 69.3% to 74.9%, that is, three in every four Brazilian households acquire information using this source (5) , including information related to health (3,6) .
Internet enables individuals to access, at any time and place, information about health (4) . However, due to this ease of access, much of the information made available online does not go through any quality assurance process (4) . This, instead of aiding in assertive decision making, leads to uninformed individuals and community, due to the excess of information and to the difficulty to evaluate its quality (3)(4)(7)(8) .
In this context, the field of eHealth (Electronic Health) emerged, which concerns the use of information and communication technologies for health (3) . This ability is necessary for individuals to evaluate the quality of the information displayed on the Internet to make assertive health decisions, that is, it is necessary for them to have adequate eHealth literacy.
The eHealth literacy is defined by Norman and Skinner (9) as "the ability to search, find, understand, and evaluate health information from electronic sources and apply the knowledge acquired to address or deal with health problems". The authors proposed, to measure this construct, the eHealth Literacy Scale (eHEALS) (9) .
The eHEALS scale was one of the first developed to measure the level of digital literacy in health. It includes eight items that aim to measure knowledge, comfort, and perceived skills of individuals at finding, evaluating, and applying electronic health information to health problems (9) .
Furthermore, it has been applied o populations such as adults with chronic diseases (26) , individuals with HIV (27) , and otorhinolaryngology (28) . However, so far, no instrument of the sort has been validated for the Brazilian population.
Therefore, efforts should be made to make available a tool that can explore and explain the structure and function of digital health literacy to instrumentalize health teams to monitor the efficacy and equity of interventions carried out digitally and indicate the impact of the use of digital means in the making of health decisions, to contribute for the adaptation of care strategies that use this resource.

OBJECTIVE
To translate and adapt the eHealth Literacy Scale for the cultural reality of Brazil and to evaluate the psychometric properties of its Brazilian Portuguese version.

Ethical aspects
The research project was approved by the Research Ethics Committee of a teaching institution according to the recommendations of Resolution 466/2012 from the National Council of Health. All participants signed the Free and Informed Consent Form.

Design, period, and place of study
A cross-sectional study was carried out with individuals who lived in the nearby areas of six Family Health Units (USF) located in a medium-sized city in the countryside of the state of São Paulo. The study was carried out from March to October 2019.

Population or sample; criteria of inclusion and exclusion
The population of the study was formed by individuals from 18 to 80 years old, who lived in the surroundings of six USFs. Those who presented mental and/or cognitive problems, with a medical diagnosis from the USF that stated their inability to respond to the instruments, were excluded. The pre-test was carried out with 50 individuals who lived near one of the USFs. After the instrument was filled in with pen and paper through a self-application, the researcher read the entire instrument with them, to search for difficulties of the respondents in comprehending specific words, questions, and/or responses. In the test stage of the final version of the instrument, it was also self-applied (like the original study) in a sample that included 502 individuals from six USFs, with a mean of 80-90 per USF. In both moments, the participants were randomly selected, had from 18 to 80 years old, and were addressed in their residences for data collection.

Instrument
The instrument is a scale with eight Likert items that varied from 1 (entirely disagree) to 5 (entirely agree), and its total score could vary from 8 to 40. The higher the score, the higher the level of digital health literacy (9) . There were also two introductory eHEALS questions, which are: "How much do you think internet is useful to help you in making health decisions?" (not useful, little useful, I am not sure, useful, very useful) and "How important is it for you to access the health information/resources available on the Internet?" (not important, little important, I am not sure, important, very important).

Procedures
Before the study started, the main authors of the original study were asked (9) , via e-mail, for their permission to translate and carry out a transcultural adaptation of the instrument, following international recommendations (29)(30) : (1) translation into Portuguese with semantic, idiomatic, and conceptual equivalence; (2) back-translation by qualified professionals; (3) specialist committee for the multidisciplinary revision of all translations and back-translations; (4) pre-test to evaluate the equivalence; (5) adjustments when needed. The original eHEALS scale was translated from English to Brazilian Portuguese by two English teachers: one of them had knowledge about the research, while another was a researcher from the health field with knowledge about the English language. The version that found a consensus was, then, translated back into English by two translators whose native language was English and who did not participate in the first stage of the translation. Then, a specialist committee was formed by six professionals from the field of health who had experiences in the field of health literacy and a high level of proficiency in the English language. Their objective was to evaluate the process and propose a final version for the document. The main author of the original instrument was contacted to clarify any doubts about the meaning of some issues and to propose modifications, to generate an instrument whose semantic validity was adequate to our reality. All questions from the instrument were found to be valid by the researchers to measure the construct. When the translated versions were compared, there were small differences in the translation of the title ("electronic literacy in health" vs. "digital literacy in health ") and in some words from the first and second questions, which did not change the meaning of the sentences and were equalized by consensus after a meeting. Translation of only one item of the instrument raised doubt: health resources. After the original author was contacted, the researchers decided to include a sentence in the disclaimer of the instrument, explaining the meaning of the term. This was the only change in the instrument that was more relevant.
In the pre-test stage, there was no need for changes in the instrument, since less than 10% of interviewees reported doubts about any item (30) .

Analysis of results and statistics
Contemporary psychometry, especially after the concept of evidence of validity (31) has required extensive testing and the integration of many stages in the several stages of instrument validation. Therefore, our analysis was based on this concept, with multiple indicators in the search for evidences of the internal structure validity, integration the three most used techniques in this stage: exploratory factor analysis (EFA -unrestricted model); confirmatory factor analysis (CFA -restricted model); and item response theory (IRT).
The dimensionality of the instrument was tested using the robust parallel analysis (RPA) through the optimal implementation of parallel analysis(PA) with the minimum rank factor analysis, which minimizes the common residue variance (32) . The robustness of the test was verified by associating a bootstrap and extrapolating the sample to 5,000. The polychoric matrix was estimated using Bayes modal estimation (33) . The dimensionality of the exploratory factor analysis (unrestricted model) was tested by a parallel analysis, which is considered to be one of the most robust and precise techniques to test dimensionality (34) . Factors were extracted using the RULS technique (Robust Unweighted Least Squares), which reduces the residue from the matrixes (35) and is more robust for non-normal data (36) . If the instrument is found to be multidimensional, the Promax rotation is used, which is a non-orthogonal (37) technique, the most appropriate for latent psychosocial variables (38) . In addition, the following were adopted as indicators to evaluate unidimensionality (39) : UNICO (Unidimensional Congruence > 0.95), ECV (Explained Common Variance > 0.80), and MIREAL (Mean of Item Residual Absolute Loadings < 0.30).
Validation techniques are recommended to increase their precision and the quality of their instruments (40) , bringing more influence to the model (41) . Consequently, the technique Normal-Ogive Graded Response Model (42) was used to evaluate the adjustment of factor loadings. The index of discrimination of item (a) was adopted to corroborate the exploratory factor analysis, since it measures the strength of the association between the item and the latent variable (43) and its interpretation is similar to the factor loadings from the EFA (44) . This study followed the recommendation according to which that an a below 0.65 is considered to have low discriminatory power; with 0.65 to 1.34 indicating moderate discriminatory power; from 1.35 to 1.69 to indicate a high discriminatory power; while an a above 1.70 was found to indicate a very high discriminatory power (45) .
Regarding the quality parameters of the instruments, the explained variation of the instrument must be around 60% (38) . The factor loadings of 0.30 are recommended when the sample has at least 300 individuals (38) , but the model is recommended to search factor loadings above 0.50 (46) ; the commonalities must have values above 0.40 (46) . Keeping or removing an item in the model depends on the magnitude of the commonalities, of factor loadings, on the size of the sample and on the degree to which the item manages to measure the factor of the nonexistence of cross-loading and Heywood cases. The reliability of the instrument was evaluated using the indicators Cronbach's alpha (47) and McDonald's omega (ω) (48) . Two indicators were adopted to increase the reliability of the interpretation. Values above ≥ 0.7 in the reliability indexes have been considered to be adequate (49) .
The factor loadings and the predictive power of the item (R 2 ) were considered to be fitness indexes for the confirmatory factor analysis. For the goodness-of-fit, a robust mean and variance-adjusted chi square was used. will be good or useful in practice, since satisfactory solution indexes can be obtained from low-quality items (39) . The analysis were carried out using the SPSS 23, AMOS 23, and Factor 10.10.1.

RESULTS
The mean age of the participants was 39.3 years old (±13.3). Only 9.3% were above 60 years old, and 65.3% were females. Additionally, 255 (50.8%) individuals had family income of up to two minimum wages. 402 (80.1%) had completed at least elementary school. Regarding how useful respondents thought internet was, 49.6% stated useful/very useful, while 24.3% were uncertain. Regarding the importance attributed by people to the ability to access health information/resources on the Internet, 52.6% thought it was important/very important, and 22.1% were uncertain.
Chart 1 shows the content of the final adapted version of the eHEALS-Br scale. Table 1 shows the mean values for each item of the instrument eHealth Literacy Scale -Brazilian scale (eHEALS-Br).
It was found that the mean values varied from 2.82 (Item 8) to 3.46 (Item 5). The mean total score of the scale eHEALS-Br for the population evaluated was 25.1 (±8,1).
The fitness analysis of the sample for the factor analyses led to a matrix determinant of (0.00013), Kaiser Meyer and Olkin (0.90) and Bartlett's test of sphericity (4443.7; p < 0.0001). The polychoric correlations of the items varied from 0.60 to 0.93. All indicators suggest the data is of good quality for the factor analysis.
The dimensionality calculated through APR showed only one dimensions with an eigenvalue of 5.86, leading to an explained variation of 81.79%. The unidimensionality of the model was confirmed by the values of UNICO (0.99), ECV (0.93), and MIREAL (0.21). As a result, it was not necessary to rotate the model.
The factor loadings settled between 0.75 and 0.90, with commonality in the range from 0.57 and 0.81 and discrimination of the item from 1.16 to 2.12, showing that the items measure the latent variable. The reliability analysis showed an alpha of 0.95 and an omega of 0.95. Furthermore, the GH index was below 0.80, indicate the stability of the model in other populations and sub-samples, with latent G-H of 0.96 and observed G-H of 0.90.
In Table 2 are presented the values of factor loadings, commonality, and item discrimination.
The index of the quality and effectiveness of score estimates also had adequate levels: Factor Determinacy Index (FDI) = 0.980, EAP marginal reliability = 0.961, sensitivity ratio (SR) = 4.955 and expected percentage of true differences (EPTD) = 96.1%. These indexes show that the score of the instrument is consistent and is not established by chance or randomly (44) .

Chart 1 -Final adapted version of the eHEALS-Br
I would like to know your opinion and your experience with the use of the internet to search for information about health, according to the questions below. For each statement, mark the answer that best reflects your opinion and your current experience. I would also like to clarify that the term "health resources", in the questions below, refers to both websites and to applications related to health.
I know which health resources are available on the Internet. 1) … Entirely disagree 2) … partially disagree 3) … I am not sure 4) … Partially agree 5) … Entirely agree I know where to find useful health resources on the internet. 1) … Entirely disagree 2) … partially disagree 3) … I am not sure 4) … Partially agree 5) … Entirely agree I know how to find useful health resources on the internet. 1) … Entirely disagree 2) … partially disagree 3) … I am not sure 4) … Partially agree 5) … Entirely agree I know how to use the Internet to clarify doubts about health. 1) … Entirely disagree 2) … partially disagree 3) … I am not sure 4) … Partially agree 5) … Entirely agree I know how to use the information about health I find on the internet to help me. 1) … Entirely disagree 2) … partially disagree 3) … I am not sure 4) … Partially agree 5) Entirely agree 5) … Entirely agree I have the skills I need to evaluate the health resources I find on the internet. 1) …Entirely disagree 2)… partially disagree 3) … I am not sure 4) … Partially agree 5)… Entirely agree I can perceive which health resources have a high quality and which have low quality on the Internet. 1) …Entirely disagree 2)… partially disagree 3) … I am not sure 4) … Partially agree 5)… Entirely agree I feel safe using information from the Internet to make health-related decisions. Some studies, however, found a two-factor structure (10,20) , while others found that the model was only found to be adequately fit when the score of the eHEALS instrument was adjusted in a three-factor model (26) . The unidimensional factor model had an explained variation of 81.79%, higher than found in the original Norman and Skinner study (56%) (9) . Other studies also found a variation from 47.80 to 75.81% (11)(12)(13)(14)(15)(16) , although it was lower than the value found by Ma and Wu (18) in a rural population in China (91.8%).
The mean total score of the instrument (25.1) was lower than that found in adults in Hungary (29.2) (21) and similar to the result for elders in Poland (25.2) (15) . Corroborating the findings from other countries (7,11,15,18) , the eight question from the eHEALS-Br instrument had the highest mean among the items of the instrument, showing that people do not feel that safe when using information from the internet to make health-related decisions, when compared to other digital health literacy skills.
Although other tools to evaluate digital literacy in health have been proposed, and despite the fact that the eHEALS evaluates the perception of the individual about health information from electronic sources, as opposed to their observed skills of searching, finding, understanding, and evaluating this information, it is one of the most used instruments to evaluate this construct, due to its good psychometric properties and ease of application (8,15) . Furthermore, as far as we know, this is the first study that evaluated the psychometric properties of the instrument in Primary Health Care users.
Considering the scarcity of instruments that are specific for measuring the digital literacy in health in Portuguese, coupled with the potential of the Internet to enable the empowering of individuals and of the community (3,8) , the Brazilian version of the eHEALS instrument can be used as a tool to plan health actions in different contexts. It stands out that the situation of the Brazilian population in regard to digital literacy in health is still unknown. Therefore, it is important to apply instrument to measure the level of literacy, contributing for professionals to educate the patient with regard to the use of electronic sources according to their performance. Also, considering the worldwide situation in the COVID-19 pandemic, the digital literacy in health has been gaining relevance, since, with social isolation, many resort to electronic resources to monitor their patients. Teleconsultations became a common reality, but it is a challenge for both patients and professionals, since its success and a successful access to reliable electronic sources can be positive or negative, and demand participation from both the professional and the patient.
In this setting, the Digital Health Strategy for Brazil 2020-2028 (50) , recently published, shows the goal of expanding the Single Health System (SUS) to improve the health care offered to Brazilian people in this aspect. The National Network of Health Data, part of the Program Conecte SUS, should be established up to 2028 as a digital platform with innovation, information, and health services for the country, involving users, citizens, patients, communities, managers, professionals, and health organizations. Although this is not the focus of the document, the goal of the program requires the population to have proper digital literacy in health. Otherwise, their participation will be reduced, as will the benefits

DISCUSSION
The analyses have shown that the Brazilian Portuguese version of the eHEALS scale (eHEALSBr) was found to be adequate for all techniques (AFE, AFC, TRI) and indicators used. Although the sampling process of this study was not broad enough to represent all 18-year-old or older people from the city and from Brazil, it was found to have good psychometric properties to measure the digital literacy construct in Brazilian adults.
Concerning its reliability, the Brazilian version of the eHEALS instrument (eHEALSBr) showed high values in both criteria (α and ω = 0,95). Furthermore, the G-H index suggests that the model can be replicated to other populations and sub-samples, mitigating the potential effects of the characteristics of the sample. Although the alpha is not a good index to compare the models, it is the only common indicator between our study and others that tested the eHEALS. The values of α were higher than those found in the original study, which was carried out with adolescents in the United States (α = 0.88) (9) . The same was true for other studies with young populations, such as that of university students in Spain (α = 0.88) (12) , adolescents in Portugal (α = 0.84) (11) , and young adults in Korea (α = 0.88) (13) . It is interesting to note that higher values in the instrument, closer to the ones found in this study, were also found in researches with older populations (14,16,19) .
The factor analysis showed that the instrument had unidimensional features, similar to those of the original document in English (9) , which was corroborated by versions in other languages (  of this innovation. The instrument validated here can contribute to operationalize this strategy, providing periodical diagnoses of the digital literacy in health of those involved, with results that can be used to reprogram procedures. As a result, this study brings important contributions for the development of future investigations related to the impact of this construct in health care. Furthermore, this is the first work about the e-HEALS instrument with a robust testing framework, integrating the techniques used in the internal structure evidence stage.

Study limitations
It stands out that the family income in the sample was relatively low. Therefore, it is reasonable to assume that the access of this population to Internet and electronic devices may be limited. Still, due to the cross-sectional nature of this study, it is impossible to provide causal inferences.
Contributions for the field of nursing, health, or public policies Nursing workers are in a privileged position to use strategies of health communication that contribute for the users of the Single Health System to be safer and more autonomous. In a setting where the population is increasingly present on-line and there is a considerable dissemination of fake news, it is paramount for health workers to incorporate the use of tools that allow them to understand the impact of the use of the information made available on the internet about the behavior of patients.

CONCLUSIONS
The Brazilian version of the eHealth Literacy Scale (eHEALS-Br) showed excellent evidences of the validity of its internal structure for the assessment of the levels of digital literacy in health for adults in Brazil.