Development of a health literacy instrument related to diabetic foot

Desarrollo de un instrumento de evaluación de la literacía en salud relacionada al pie diabético

ABSTRACT

Objective:

To develop and verify the validity, reliability and interpretability of an assessment instrument of the Health Literacy regarding Diabetic Foot (HLDF).

Method:

Methodological study among people with diabetes enrolled in Family Health Strategy (FHS) program, composed of eight phases: selection of participants; development of LSPD; verification of content validity by the committee of experts; pre-test (n=20); reliability estimation (n=62): Cronbach's alpha (CA), Kappa (K) and Intraclass Correlation Coefficient (ICC), satisfactory results (≥0.60); estimation of concurrent validity (n=62); interpretability of scores: range from 0-18, if ≤14 HLDF inadequate; hypothesis test (multiple logistic regression): OR/IC95% in a sample estimated for infinite population, using SPSS.

Results:

The HLDF has adequate and fast application, being relevant as to its content and to the construct that proposes to evaluate. About the reliability CA=0.73, two items presented Kappa<0.60, ICC=0.79. About the concurrent validity, there was correlation between the HLDF and schooling (rs=0.647; p=0.000). Interpretability: 50.2% (n=112) presented inadequate HLDF. The younger ones presented lower levels of inadequate HLDF (0.96/0.93-0.99); as to those with lower scholarship (1.15/1.06-1.25) and lower family income (1.01/1.00-1.01) presented higher levels of inadequate HLDF.

Conclusion and implications for practice:

The HLDF was considered as valid, reliable and interpretable. The HLDF can identify people with more effective educational strategies needs.

Keywords:
Health Literacy; Diabetes Mellitus; Diabetic Foot; Reproducibility of Results

RESUMEN

Objetivo:

Desarrollar y verificar la validad, fiabilidad e interpretabilidad de un instrumento de evaluación que propone evaluar la Literacia en Salud en lo que concierne al Pie Diabético (LSPD).

Método:

Estudio metodológico entre personas con diabetes catastradas en Estrategias de Salud de la Familia, compuesto por ocho fases: selección de los participantes; desarrollo del LSPD; verificación de validad del contenido por el comité de expertos; pre-test (n=20); estimación de la confiabilidad (n=62); Alfa de Cronbach (AC), Kappa y Coeficiente de Correlación Intraclase (CCI); resultados satisfactorios (≥0,60); estimación de la validez concurrente (n=62); interpretabilidad de los scores: varían de 0-18 se ≥14 LSPD adecuado; prueba de hipótesis (regresión logística múltiple): OR/IC95% en una muestra estimada para población infinita, utilizando el SPSS.

Resultados:

El LSPD tiene una aplicación adecuada y rápida, siendo relevante en relación con su contenido y al constructo que propone evaluar; en relación con su confiabilidad AC=0,73; dos de los elementos presentaron Kappa<0,60; CCI=0,79. En cuanto a la validez concurrente, hubo correlación del LSPD con la escolaridad (rs=0,647; p=0,000). Interpretabilidad: 50,2% (n=112) presentaron LSPD inadecuada. Los más jóvenes presentaron niveles más bajos de LSPD inadecuada (0,96/0,93-0,99); ya aquellos con menor nivel de educación (1,15/1,06-1,25) y menor renda familiar (1,01/1,00-1,01) presentaron niveles más altos de LSPD inadecuada.

Conclusión e Implicaciones para la práctica:

El LSPD fue considerado válido, confiable e interpretable. El LSPD podrá identificar personas con necesidades de estrategias educativas más efectivas.

Palabras claves:
Alfabetización en Salud; Diabetes Mellitus; Pie diabético; Reproducibilidad de los Resultados

RESUMO

Objetivo:

Desenvolver e verificar a validade, confiabilidade e interpretabilidade de um instrumento de avaliação da Literacia em Saúde no que diz respeito ao Pé Diabético (LSPD).

Método:

Estudo metodológico entre pessoas com diabetes cadastradas na Estratégia de Saúde da Família, composto por oito fases: seleção dos participantes; desenvolvimento do LSPD; verificação da validade de conteúdo pelo comitê de especialistas; pré-teste (n=20); estimativa da confiabilidade (n=62): Alfa de Cronbach (AC), Kappa (K) e Coeficiente de Correlação Intraclasse (CCI), resultados satisfatórios (≥0,60); estimativa da validade concorrente (n=62); interpretabilidade dos escores: variam de 0-18, se ≤14 LSPD inadequada; teste de hipótese (regressão logística múltipla): OR/IC95% em uma amostra estimada para população infinita, usando o SPSS.

Resultados:

O LSPD possui aplicação adequada e rápida, sendo relevante quanto ao seu conteúdo e ao construto que propõe avaliar; quanto à confiabilidade: AC=0,73, dois itens apresentaram K<0,60; CCI=0,79. Quanto à validade concorrente, houve correlação do LSPD com a escolaridade (rs=0,647;p=0,000). Interpretabilidade: 50,2% (n=112) apresentaram LSPD inadequada. Os mais jovens apresentaram menores níveis de LSPD inadequada (0,96/0,93-0,99); já aqueles com menor escolaridade (1,15/1,06-1,25) e menor renda familiar (1,01/1,00-1,01) apresentaram maiores níveis de LSPD inadequada.

Conclusão e implicações práticas:

O LSPD foi considerado válido, confiável e interpretável. O LSPD poderá identificar pessoas com necessidades de estratégias educacionais mais efetivas.

Palavra-chave:
Alfabetização em Saúde; Diabetes Mellitus; Pé diabético; Confiabilidade e Validade

INTRODUCTION

The Diabetes Mellitus (DM) concerns the metabolic disturbances provided by the deficit in the insulin action and/or secretion and the increase of the glycemia. The estimates show that, in 2040, 23.3 million Brazilians will have the diagnosis of DM. Its high prevalence relates to the urbanization, epidemiological and nutritional transition, low frequency of physical activity, overweight and obesity, growth and increase in survival of the population, including persons with DM. In general, its late diagnosis is associated with its complications¹.

Among the DM complications, it highlights the Diabetic Foot (DF), characterized by infections, ulcers and/or destruction of the deep tissues, with slow healing in the lower limbs, especially in the feet. The DF can impact negatively on the quality of life of DM people, especially in amputation cases; can be associated with neurological abnormalities, peripheral vascularization and with metabolic changes.^2-5 The prevalence of DF among people with DM can vary from 4 to 10%, showing itself higher in countries with poor socioeconomic conditions. It should be stressed that, on worldwide level, every year 1 million people with DM has an amputated part of the leg.⁶

Low levels of Health Literacy (HL) on DM and its complications favor the manifestation of the DF. In the opposite way, good levels can eliminate or delay such manifestation due to the self-care.^7,8 In light of this, investigating the level of HL of persons with DM regarding the care of their feet can contribute to the development and implementation of strategies on prevention and care of the DF. Thus, the HL may improve the quality of life of people and promote the economy of public resources.^7,9 Then, it can be observed the need to incorporate actions that favor the development of the HL among people with DM.⁸

The HL concerns the personal, cognitive and social skills necessary for people to understand, evaluate and apply the information necessary for the maintenance of health.⁷ Negative consequences among people with DM can be associated with low level of HL. In front of this, it is important to find ways to quantify the levels of HL in this public, by means of valid and reliable instruments.^10,11

There are evaluation instruments of general and specific HL. The general estimate the proportion of persons with difficulty of HL related to general health, as an example the SAHLPA instrument (Short Assessment of Health Literacy for Portuguese-speaking Adults) that evaluates the ability of the individual to speak and understand terminologies related to general health.¹² Those of a specific nature propose detailed assessments of levels of HL in a particular health condition, such as the DF.¹³ Decisions related to health promotion, to preventive intervention and to therapy of diseases, as well as to creation or continuity of strategies and health policies may be influenced by results of investigations that used instruments for assessment of general and specific HL.^11,12 However, not all instruments that propose to assess conditions related to health present acceptable methodological qualities.^10,11 Valid, reliable instruments and easy to interpret can help health professionals and researchers in the recognition of persons who need for approaches consistent with the reality found.¹² A study Delphi showed a checklist to assess the quality of instruments that propose to measure events related to health, the COSMIN - checklist (Consensus-Based Standards for the selection of health Measurement Instruments).¹⁰

The COSMIN - checklist presents steps to identify the methodological quality of investigations that aim to assess events related to health. This tool helps to assess research instruments, either scales or questionnaires. The COSMIN - checklist presents methodologically correct parameters for the events related to health evaluation. It is composed by four domains: three for estimating the reliability, validity and responsiveness and one for assessing the interpretability.^10,11 There have not been found scientific publications of instruments for assessing the levels of HL related to DF; fact which influenced on the proposal of this research.

The aim of the study was to develop and verify the validity, reliability and interpretability of an instrument to assess the HL concerning the DF.

METHOD

This is a methodological study, carried out between 2016 and 2018. So it could be implemented, the PubMed, Scientific Electronic Library Online and Health Virtual Library were accessed in order to carry out a literature review through researches that addressed the theme HL and DF, as well as the main measuring instruments used. The "Health Literacy" e "Diabetic Foot" descriptors were used by applying the Boolean operator "and". Original studies published in English and Portuguese idioms were considered, without a time limit.

Stages of the study

Eight phases were the development of the instrument called Health Literacy for the Diabetic Foot (HLDF): 1) Selection of participants 2) Development of the HLDF (conceptual structure, definition of objectives of the construct, construction of items/response scale, selection and organization of items and structuring of the HLDF)¹⁴; 3) Validity of content; 4) Pre-test; 5) Estimate of the reliability (Cronbach´s Alpha-CA) and the test-retest (kappa-K and Intraclass Correlation Coefficient-ICC); 6) Concurrent Validity (correlation test); 7) Interpretability; 8) Hypothesis test (logistic regression).

Selection of participants

It was performed a contact with the Municipal Coordination of Family Health of a medium-sized population municipality to obtain lists with the enumeration of the Pole Units (PU) of the Family Health Strategy teams (FHS). It should be noted that the PU is a health physical unit where more than one FHS team actuate, with the offer of assistance to its public. Simple randomized drawings of the PUs were done for each step of the data collection: 1^st collection - Pre-test; 2^nd collection - Estimate of the reliability and concurrent validity; 3^rd collection - Hypothesis test.

Among the 73 existing units in the municipality, 05 were drawn. Of these, one was used for the pre-test (n=20) and other two for estimating the reliability and the concurrent (n=62); for the hypothesis test (sample estimated of 216 persons), persons with DM of other two PUs not considered previously were interviewed. The managers of the FHS of the PUs that were raffled provided lists with the name of persons with DM registered at the FHS. Participants were included until they reach the quantity pre-established for convenience sample and for sample calculation, in accordance with the step of the research.

To verify the reliability (internal consistency and reproducibility) and the concurrent validity, a sample of 62 persons with DM was used, as populations consisted of 50 to 100 participants are sufficient.^15,16 A sample of 216 persons with DM was conceived for the hypothesis test with multiple analysis in infinite population, which considered the parameters: Z=1.96/95% confidence level (1- α); p=0.50 proportion of people with diabetes; 1-p=proportion of people without diabetes; d=tolerable error of 0.07 + 10% of losses/n=196 +20 persons [possible losses] = 216).^14,17 Despite estimating a sample of 216 persons for the hypothesis test, it was searched for a greater number of participants according to the possibility of loss and the risk of not achieving the number conceived. It was regarded as loss the instruments that contained smudged or incomplete responses in three variable or more.

People age ≥ 18 years, registered in the FHS and with DM diagnosis, according to information from the FHS were included in the study. We excluded people with three or more comorbidities, who had native language other than that of the Portuguese, presented vision or hearing problems reported or perceived by the researcher at the time of the approach for the data collection, as well as those who presented intoxication by drugs or alcohol at the time of the interview¹⁴. The cognitive impairment, defined through the Mini-Mental State Examination (MMSE), was the other exclusion criterion verified among people aged 60 years and, as this group demonstrates high risk of developing mental confusion and of not receiving a correct diagnosis.¹⁸

Development of the HLDF

The choice of the content or words used in the HLDF instrument was defined by the literature review^{3-6,8,9,19-23}; there were selected words or expressions that more were repeated in the studies, classic expressions about the subject and that represented the DF condition. Moreover, we considered the theoretical model proposed by Sørensen, which describes aspects related to HL⁷ (personal, cognitive and social skills for accessing, understand, assess and apply information related to health) and the SAHLPA.¹² This is composed of 18 threesome of words related to the access and the understanding of factors associated with general health, being that, among every threesome of words, one central expression should be associated with one of the other two for defining the level of HL¹⁴. It is added 01 point for each correct association. The result may vary between 0 and 18 points, being the cutoff point ≤ 14, which suggests problems in the LS.¹² In this study, we used the SAHLPA as a reference for structuring the HLD.

Content validity

The HLDF was structurally organized presenting title, instructions and response scale. The adequacy and consistency of HLDF items were analyzed in the content validity. The quality of HLDF, especially if were representative of the construct, has been evaluated by a committee of 10 professionals/judges. The judges presented different degrees and experiences, which were selected by convenience (Dental Surgeon, Physical Educator, Nurse, Pharmacist, Physiotherapist, Psychologist, Dietitian, General Physician, Endocrinologist Physician and Healthcare Technician). The judges presented publications related to the subject and or had technical/methodological knowledge concerning the development of instruments for assessing conditions related to health, so as some worked in the prevention, treatment or in the recovery of the neuropathic complications of DM and others in conducting scientific researches.^24-26 The literature diverges in relation to quantity of judges necessary to compose a committee of specialists. As this number can vary from 3 to 20 members²⁵, it was decided to invite 10 judges, who invited an invitation to participate in the content validity of the instrument. After the acceptance and with seven days in advance an explanatory letter has been sent containing the objective of the step: assessing the representativeness of the questionnaire items regarding what was intended to be measured, HL as to DF. We conducted a focus group for discussing the items of the instrument, and all were oriented to suggest adaptations, inclusions and/or exclusions of items.²⁶ The Chart 1 shows the final version of HLDF.

Chart 1
Final version of the instrument Health Literacyas to Diabetic Foot – LSPD.

Final version of HLDF
MAIN WORD	WORDS OF ASSOCIATION
1. DIABETIC FOOT	O DISEASE	O Body part	O I do not know
2. HYPERGLYCEMIA	O HIGH	O Low	O I do not know
3. INSENSITIVITY	O Thought	O LOSS	O I do not know
4. NEUROPATHY	O NERVOS PERIFÉRICOS	O Estressado	O I do not know
5. TINGLING	O Sleeping	O NUMBNESS	O I do not know
6. AMPUTATE	O REMOVE	O Walking	O I do not know
7. INSPECTION	O DAILY	O Monthly	O I do not know
8. ULCER	O Malaise	O WOUND	O I do not know
9. INFECTION	O COMPLICATION	O Injection	O I do not know
10. CLEANLINESS	O CARE	O Aesthetics	O I do not know
11. SOCKS	O Synthetic	O COTTON	O I do not know
12. SHOE	O Fair	O COMFORTABLE	O I do not know
13. WALK	O Barefoot	O SHOE	O I do not know
14. WATER	O Hot	O ROOM TEMPERATURE	O I do not know
15. SOAP	O NEUTRAL	O Powder soap	O I do not know
16. DRY	O Hair	O BETWEEN THE TOES	O I do not know
17. NAILS	O SQUARED	O Rounded	O I do not know
18. MYCOSIS	O Normal	O TREATING	O I do not know

¹²

Pre-test

The pre-test was applied by four interviewers (three scientific initiation students of the physiotherapy course and one nurse specialist in Family Health) trained for that procedure. The participants were interviewed in their own residence, identified from the registration in the FHS. Initially, information about gender and age were requested; next, printed plates were used for application of the HLDF instrument. The main terms were located in the top, typed in bold and the two possibilities of association were described just below without highlighting them and with similar size and font. The interviewer carried out the reading of the main term, and after, questioned the participant about which word would be associated correctly with the term read. It was considered as 1 point for every word associated correctly, with result ranging from zero to 18 points. Prior to the interviews, the participants were asked not trying to guess the word with correct association; in this case the participants could say, "I do not know" if they did not know to associate correctly the item. The words of correct association were highlighted in uppercase in the HLDF instrument (capital letters). This version was not presented to the participants, since such formatting aimed to facilitate the visualization and the consolidation of results only for the interviewer.

The average time for the HLDF instrument application was 4.36 minutes (minimum: 3 min.; maximum: 6.10 min.). After the pre-test, all interviewers were invited to participate in a last meeting with the committee of specialists to relate their perceptions regarding HLDF application. This meeting was conducted by the coordinator of the project through focus group within the activities of scientific initiation of the State University of Montes Claros. Since no need for an amendment in the HLDF instrument was found, this version was released for estimating the reliability.

Reliability estimation

The participants were interviewed in their own residence. Firstly, a socioeconomic and demographic questionnaire was applied for obtaining personal data (gender, age, schooling, marital status, self-declared skin color/race, family income, spending on medication). Subsequently, the HLDF instrument was applied in the same method used for the pre-test (printed plates). The reliability was estimated through internal consistency and of the reproducibility; the internal consistency was measured by the Cronbach´s Alpha calculation (CA), which assumes values between 0 and 1. The closer the 1, the more reliable, considering acceptable values ≥ 0.7.²⁷ In the estimate the reproducibility, the test-retest method was applied through interviews conducted in a period of time from 3 to 7 days.^15,16 The occurrence of similar results was found, by measuring the event in the same individuals in different situations.²⁸ Then, the Kappa (K) coefficient was applied for each of its items with the following interpretation: K < 0.00 = almost nonexistent; 0-0.19 = small; 0-0.39 = unsatisfactory; 0.40-0.59 = moderate; 0.60-0.79 = substantial; 0.80-1.00 = almost perfect.^14,29 On the other hand, the reproducibility estimation for the total scores, considered the Intraclass Correlation Coefficient (ICC).

Concurrent validity

The concurrent validity was estimated through the correlation between the scores obtained in the HLDF and the schooling of the participants. We opted for the adequate test to distribution of normality of the sample: Pearson´s correlation or Spearman; the choice for this correlation was defined through the statistical test of normality (Kolmogorov-Smirnov Tests), with a significance level of 5%.

Interpretability

Regarding the interpretability, the cut-off point of the HLDF instrument was defined by means of the Confidence Interval (CI). Thus, participants who presented score equal or below the CI inferior limit of the mean (values ≤ 14) were considered as with inadequate Literacy as to the access, understanding and assessment of information related to the DF. In addition, the HLDF was conceived on the basis of the structure and the scales of the SAHLPA answer¹², which contains 18 items and the same method of application and scoring. In its validity process, the SAHLPA reached the same cut-off point (values ≤ 14 suggest Inadequate literacy).¹²

Hypothesis test

For the hypothesis test, the data collection occurred by applying the same method used in the pre-test and test-retest. A socioeconomic and demographic questionnaire was applied for obtaining personal data (gender, age, schooling, marital status, self-declared skin color/race, family income, spending on medication) and the HLDF instrument was applied by means of printed plates. Persons with DM users of two PUs of the FHS participated in this step, disregarding the participants in the pre-test and the test-retest. The same inclusion and exclusion criteria applied in the pre-test and test-retest phases were considered in the phase of the hypothesis test.

The construct validity includes three estimates (Structural Validity, Hypothesis Test and Cross-Cultural Validity), on the other hand the one of criterion includes the estimates of the concurrent and predictive validities.^10,11 The Structural Validity was not estimated since it is not applicable to binary variables; the Cross-Cultural Validity is not necessary, since the HLDF was developed in the native language of the participants; the predictive validity in turn could not be estimated because of the inexistence of golden standard.¹⁰ Therefore, the criterion validity (concurrent) and the hypothesis test were estimated.

Statistical analysis

In the descriptive analysis, the absolute and relative frequencies were estimated, for the categorical variables, and the mean, standard-deviation and maximum and minimum values, for the continuum variables. The association between the independent variables (gender, age, schooling, marital status, self-declared skin color/race and family income) and the dependent (Literacy: adequate/inadequate) was verified through the chi-square test, with odds ratio values and 95% confidence interval. The variables associated with the dependent variable (p≤0.20) in the bivariate analysis were included in the multiple analysis. The logistic regression was applied for estimating the multiple model. The final model was adjusted, maintaining the variables associated with p < 0.05. For the statistical analysis, we used the Statistical Package for the Social Sciences - SPSS, version 24.0 and Excel programs for Windows. This study is a subproject of a project approved by the Research Ethics Committee of the State University of Montes Claros (Unimontes), opinion No. 764.743/2014.

RESULTS

In the literature review 11 original studies published between 2013 and 2017 were selected. These productions served as the basis for constructing the items of the HLDF. No valid and reliable instruments that intended to evaluate the HL as to the DF were found.

Participants´ characterization of the pre-test

Twenty persons with DM participated in the pre-test. Among these, 65% (n=13) were female with a mean age of 54.05 (DP=10.13; minimum: 35 years of age; maximum: 73 years of age). Six participants had 60 years of age or older, all of them presented preserved cognitive state.¹⁸

Participants characterization of the test-retest and hypothesis test

Of the 62 participants in the test-retest, 83.9% (n=52) were female. The average schooling was 5.63 (DP=3.99), ranging from 0 to 25 years of study. Age ranged from 29 to 77 years (Mean=54.9; DP=9.97). Among the interviewers, 18 were 60 years of age or older; the MEEM test was applied and all of them presented preserved cognitive state.¹⁸

For the implementation of the hypothesis test, a sample of 216 persons with DM was estimated, however, a greater number of participants has been reached in data collection (n=282); favorable situation with regard to the possibility of losses and to the risk of not achieving the idealized number. Of the 282 participants in the hypothesis test phase, 67% (n=189) were female. Age ranged from 22 to 92 years, with a mean age of 61.02 years (DP=11.321; IC95%=59.62-62.52). The family income ranged from R$ 0.00 to R$ 12,000.00, the average was R$ 2,216.73 (DP=1,854.13; IC95%=1,995.62-2,465.55). The average schooling was 7.55 (DP=4.34; IC95%=7.03-8.05), ranging from 0 to 28 years of study. The average expenditure on medicines was R$ 68.24 (DP=138.18; IC95%=52.51-85.43), ranged from R$ 0.00 to R$ 1,000.00, of which 52.9% (n=148) report these spending (Table 1). Among the participants, 58.4% (164) were 60 years old or more; all of them presented preserved cognitive status, confirmed by the MEEM.¹⁸

Table 1
Socioeconomic and demographic information of persons with DM of PUs of the FHS of a medium-sized population municipality, Minas Gerais, Brazil, 2016. (n=62/n^¥=282)

Variables	n	%	n¥	%
Gender
Female	52	83.9	189	67.0
Male	10	16.1	93	33.0
Stratified Age*
22 to 54	--	--	76	27.0
55 to 61	--	--	63	22.4
62 to 68	--	--	74	26.3
69 to 92	--	--	68	24.3
Schooling (years of study completed)*
0	05	08.1	17	06.1
1 to 4	25	40.3	71	25.4
5 to 8	19	30.6	85	30.4
9 to 11	10	16.1	51	18.2
12 or more	03	04.8	56	20.0
Marital status*ed
Married(female)/Stable union	--	--	169	61
Single(female)/Widowed(female)/Divorced/separated(female)	--	--	108	39
Self-declared skin color/race*
White	--	--	85	31.4
Yellow	--	--	14	05.2
Black	--	--	42	15.5
Brown	--	--	128	47.2
Indigenous	--	--	02	0.7
Family income*
Up to R$ 937.00	--	--	201	76.7
Above R$ 937.00	--	--	61	23.3

• -- Not applicable.
• ^*
  Number of respondents less than the number of participants.

Validity of the HLDF instrument content

After the content analysis by the committee of specialists, it was found that the HLDF has an adequate application method. In the same way, it was considered as relevant concerning its content and with regard to the construct which seeks to evaluate. Modifications in the structure or in the response scales of the HLDF instrument were not necessary after the pre-test application.

Estimate of the HLDF instrument reliability

The CA (test to evaluate the internal consistency) estimated was 0.73. Two words (Neuropathy and Shoe) presented Kappa lower than 0.60, among the 18 words of the HLDF in the evaluation of the reproducibility (Table 2). The ICC estimated for the total score of the HLDF was 0.79 (p=0.000).

Table 2
Levels of agreement (Simple Kappa Coefficient) for associations of words and/or of the instrument terms Health Literacy to the Diabetic Foot among persons with DM of PUs of the FHS of a medium-sized population municipality, Minas Gerais, Brazil, 2016. (n=62).

Main word/words of associations	Kappa* Reproducibility
Diabetic Foot/DISEASE; Body part	0.65
Hyperglycemia/HIGH; Low	0.90
Insensitivity/Thought; LOSS	0.72
Neuropathy/PERIPHERAL NERVES; stressed	0.53
Tingle/Sleep; NUMBNESS	0.61
Amputate/REMOVING; Walking	0.86
Inspection/DAILY; Monthly	0.82
Ulcer/Malaise; WOUND	0.68
Infection/COMPLICATION; Injection	0.62
Clean/CARE; Aesthetic	1.00
Socks/Synthetic; COTTON	1.00
Shoe/Right; COMFORTABLE	0.30
Walk/Barefoot; SHOE	1.00
Water/Hot; ROOM TEMPERATURE	0.84
Soap/NEUTRAL; Soap powder	0.80
Dry/Hair; BETWEEN THE TOES	1.00
Nails/SQUARE; Rounded	0.91
Micosis/Normal; TREATING	1.00

• ^*
  p<0.05 for all items.

Concurrent Validity

The normality test (Kolmogorov-Smirnov Tests) presented p=0.000. Thus, we opted for the Spearman´s correlation coefficient (non- parametric) to verify an association between HLDF and the participants´ schooling. After applying the Spearman´s correlation test, it was found a positive correlation of the HLDF with years of schooling (Spearman´s Correlation=0.647; p=0.000). In this regard, it was noted that high levels of HLDF are related to high levels of schooling.

Interpretability of the test-retest

The mean score of the HLDF reached in the test-retest (n=62) was 15.40 (DP=3.30; IC95%=14.56-16.24). Among the 62 interviewees, 22 (35.5%) presented score less than or equal to 14, presenting inadequate Literacy level.

Interpretability of the hypothesis test

Among the 282 participants of the hypothesis test, 223 replied to the HLDF in full, whose score ranged from 0 to18 (Mean=14.18; DP=2.39; IC95%=13.82-14.49); 112 participants (50.2%) presented inadequate Literacy level (score ≤14). The main word "Walk" was the most accurate association among the participants: 274 (97.2%) replied correctly "Shoe". On the other hand, the word "Nails" was the association that presented the biggest mistake/do not know: 155 (56.4%) participants replied erroneously "Rounded" (Table 3).

Table 3
Frequency of successes and mistakes/do not know of the application of the instrument Health Literacy to the Diabetic Foot of persons with DM of PUs of the FHS of a medium-sized population municipality, Minas Gerais, Brazil, 2016. (n=282)

Main Word/Correct association	N	%
Diabetic foot/Disease*
Success	191	68.2
Mistake/Do not know	89	31.8
Hyperglycemia/High*
Success	175	63.9
Mistake/Do not know	99	36.1
Insensitivity/Loss*
Success	186	68.6
Mistake/Do not know	85	31.4
Neuropathy/Peripheral nerves*
Success	120	46.9
Mistake/Do not know	136	53.1
Tingling/Numbness*
Success	265	94.3
Mistake/Do not know	16	05.7
Amputate/Remove*
Success	249	89.2
Mistake/Do not know	30	10.8
Inspection/Daily*
Success	177	70.8
Mistake/Do not know	73	29.2
Ulcer/Wound*
Success	253	90.4
Mistake/Do not know	27	9.6
Infection/Complication*
Success	241	86,1
Mistake/Do not know	39	13.9
Clean/Care
Success	265	94.0
Mistake/Do not know	17	06.0
Socks/Cotton*
Success	227	81.7
Mistake/Do not know	51	18.3
Shoe/Comfortable
Success	271	96.1
Mistake/Do not know	11	03.9
Walk/Shoe
Success	274	97.2
Mistake/Do not know	08	02.8
Water/Room temperature*
Success	264	94.0
Mistake/Do not know	17	06.0
Soap/Neutral*
Success	244	87.5
Mistake/Do not know	35	12.5
Dry/Among the toes*
Success	160	57.1
Mistake/Do not know	120	42.9
Nails/Squared*
Success	120	43.6
Mistake/Do not know	155	56.4
Micosis/Treat
Success	268	95.0
Mistake/Do not know	14	05.0

• ^*
  Number of respondents less than the number of participants.

Results of the statistical analysis of the hypothesis test

In the bivariate analysis, only the variables age, schooling and family income had been associated with the dependent variable (HLDF - adequate/inadequate). In the model adjusted by the statistically significant variables (p≤0.05), it was noted that, the greater the participant´s age, the greater its inadequate level of Literacy; it was possible to confirm that, the lower its schooling and family income, the greater its inadequate level of Literacy (Table 4).

Table 4
Bivariate analysis and adjusted model of the association between the inadequate HLDF and the independent variables among users of PUs of the FHS of a medium-sized population municipality, Minas Gerais, Brazil, 2016. (n=203 Adjusted model/p ≤ 0.05).

Variable	Literacy				Bivariate analysis			Adjusted model
	Adequate		Inadequate
Age	n	%	n	%	OR	IC95%	p	OR	IC95%	p
Discrete variable	--	--	--	--	0.96	0.931-0.981	0.001	0.96	0.93-0.99	0.006
Gender
Female	71	33.3	67	31.5	1			--	--	--
Male	36	16.9	39	18.3	1.19	0.68-2.06	0.541	--	--	--
Schooling
Discrete variable	--	--	--	--	1.22	1.13-1.31	0.000	1.15	1.06-1.25	0.001
Self-declared skin color/race
White	34	16.0	37	17.4	1			--	--	--
Yellow/black/brownindigenous	73	34.3	69	32.3	0.87	0.49-1.54	0.628	--	--	--
Family income
Discrete variable	--	--	--	--	1.01	1.00-1.01	0.010	1.01	1.00-1.01	0.037

• -- Not applicable. Hosmer-Lemeshow Gross Model: p=0.692 and Adjusted Model: p=0.920.

DISCUSSION

The self-care of persons with DM is essential in the prevention of complications related to DF.^9,19,20 The promotion of actions of education in health can increment the self-care, minimizing eliminating complications.²¹ Then, it highlights the importance of the instruments that propose an evaluation of the HLDF. The results of assessments of the HL levels enable strategies and public policies in the fight against complications¹¹ such as the DF. However, all methodological parameters concerning the validity and the reliability of an instrument must be considered in the decisions regarding the clinical practice or the public policies, considering the circumstances, the population involved, the type and the objective of the study.³⁰

Concerning the reliability of the HLDF, the analysis were considered as satisfactory, so that its use was recommended. The internal consistency/CA=0.73 was adequate;^25,31 the reproducibility obtained was satisfactory, only two words showed K ≤ 0.60 (Neuropathy; Shoe). Similar results were observed in studies that assessed different events related to health.^12,32-34 The estimate unsatisfactory, observed in the word "Neuropathy", can be a consequence of the application of the test that motivated the participants to identify the meaning of this word during the test-retest interval. Another similar situation may have occurred in the word "Shoe"; the persons with diabetes may have understood the word "Right" as benefits and safety for the feet, causing an erroneous understanding with regard to care with the DF. The ICC=0.79 was also adequate; a close result was previously recorded.¹² It is intended to minimize the influence of external factors during the interval of applications of HLDF, since the interval was 3-7 days. However, there is likelihood of mistakes in the estimate of the reproducibility regarded to the natural changes during such interval.³⁵

Concerning the concurrent validity, it was noted that high levels of HLDF are related to high levels of schooling, showing a positive correlation (Spearman Correlation=0.647; p=0.000), result previously recorded.^12,36-40 It´s because the school education and the understanding of the health condition make healthy practices possible. Even though, it is necessary a dialogue between the health professional and the patient to stimulate possible changes of habits and attitudes, in a language consistent with its socio-economic and educative language.²¹

The prevalence of the adequate HL related to DF was expected²². The HLDF may be a consequence of the access and the understanding of the information related to health, but not necessarily this knowledge is followed by healthy assessments and choices, that is, adequate behavior. As for the hypothesis test, it should be noted that the "n" assessed was higher than the one initially estimated, since the own losses of the multiple analysis were considered in the planning. Therefore, although it has been estimated an n=216, opted to assess 282 persons with DM with the aim of obtaining in the multiple analysis an "n" close to the estimated. In the investigation, the "n'" estimated was 216 and the "n" of the multiple analysis was 203, similar values. The estimate aimed of dispensing the imputation of missing data.^41,42 The association between schooling, family income and age with HLDF has been confirmed, since it is observed that, the lower the age, the higher the schooling and income of the individual and higher is its level of HL.^43-46 The low schooling and the advanced age are predominant characteristics among users of the FHS, among which the change of behavior is more difficult.⁴³ Thus, the health team of the FHS has a challenge of identifying needs and create tools to facilitate the application of strategies of promotion and health prevention in their daily. We must also consider that the spirituality and the religion are a source of tranquility, holding and energy for many people in the care related to DF; considering this issue in the treatment can promote hope and the treatment adherence, observing, consequently, improvement in the clinical condition and person satisfaction with the DF.²³

It is considered as the study limitation, the fact that the HLDF does not include the evaluation and information application related to the DF, according to Sorensen´s proposal,⁷ since it opted to work with the SAHLPA methodology.¹² In addition, the hypothesis test was conducted in a simple randomized non-probabilistic sample, situation that would allow population inferences. On the other hand, the study allowed to discriminate different levels of Literacy related to DF, since the main study objective was to discuss about the HLDF quality. Long instruments may generate bias of measurement, however the HLDF has a dynamic application, which favors reliable responses.²⁸

CONCLUSION

The HLDF demonstrated dynamic applicability, fast and of easy understanding. It was considered as valid, reliable and interpretable. It can be used to evaluate the HL with respect to the DF. It was noted that younger people with a diagnosis of DM presented lower levels of inadequate HLDF and with small schooling and small family income presented greater levels of inadequate HLDF. This instrument could help researchers and health professionals, in special, nurses, in the identification of persons with greater communication needs. It might influence the creation and/or maintenance of educational strategies compatible with the reality of persons assessed with DM, reducing the probability of complications, costs with treatment and the morbimortality.

• FINANCIAL SUPPORT
  Foundation for Supporting Research of Minas Gerais State (FAPEMIG), Process: BIP-00384-16. Brazilian National Council for Scientific and Technological Development (CNPq), Process: 456224/2014-9. United Faculties of the North of Minas (FUNORTE) - Scholarship of Scientific Initiation internal process of selection.

ACKNOWLEDGMENTS

Logistical support of the United Faculties of the North of Minas (FUNORTE) and of the State University of Montes Claros (Unimontes).

REFERENCES

• ¹
  Sociedade Brasileira de Diabetes. Diretrizes da Sociedade Brasileira de Diabetes 2017-2018. José Egídio Paulo de Oliveira, Renan Magalhães Montenegro Junior, Sérgio Vencio, organizadores. São Paulo: Editora Clannad; 2017.
• ²
  Brasil. Grupo de Trabalho Internacional sobre Pé Diabético. Consenso Internacional sobre Pé Diabético; publicado sob a direção de Hermelinda Cordeiro Pedrosa; tradução de Ana Claudia de Andrade, Hermelinda Cordeiro Pedrosa. Brasília: Secretaria de Estado de Saúde do Distrito Federal; 2001.
• ³
  Pedras S, Carvalho R, Pereira MG. Quality of life: non-amputee patients with diabetic foot ulcer versus amputee patients. Psic Saúde & Doenças [Internet]. 2016 apr; [cited 07 ago 2018]; 17(1):89-96. Available from: http://www.scielo.mec.pt/scielo.php?script=sci_arttext&pid=S1645-00862016000100013&lng=pt
  » http://www.scielo.mec.pt/scielo.php?script=sci_arttext&pid=S1645-00862016000100013&lng=pt
• ⁴
  Almeida SA, Silveira MM, Santo PFE, Pereira RC, Salomé GM. Avaliação da qualidade de vida em pacientes com diabetes mellitus e pé ulcerado. Rev Bras Cir Plást [Internet]. 2013 mar; [cited 02 apr 2018]; 28(1):142-6. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1983-51752013000100024
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1983-51752013000100024
• ⁵
  Amaral Júnior AH, Amaral LAH, Bastos MG, Nascimento LC, Alves MJM, Andrade MAP. Prevenção de lesões de membros inferiores e redução da morbidade em pacientes diabéticos. Rev Bras Ortop [Internet]. 2014 oct; [cited 10 feb 2018]; 49(5):482-87. Available from: http://www.scielo.br/scielo.php?pid=S0102-36162014000500482&script=sci_arttext&tlng=pt
  » http://www.scielo.br/scielo.php?pid=S0102-36162014000500482&script=sci_arttext&tlng=pt
• ⁶
  Boulton AJ, Vileikyte L, Ragnarson-Tennvall G, Apelqvist J. The global burden of diabetic foot disease. The Lancet [Internet]. 2005 nov; [cited 26 mar 2019]; 366(9498):1719-24. Available from: https://www.ncbi.nlm.nih.gov/pubmed/16291066
  » https://www.ncbi.nlm.nih.gov/pubmed/16291066
• ⁷
  Sørensen K, Broucke SV, Fullam J, Doyle G, Pelikan J, Slonska Z, et al. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health [Internet]. 2012 jan; [cited 10 feb 2018]; 12(1):80. Available from: https://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-12-80
  » https://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-12-80
• ⁸
  Sá Policarpo N, Moura JRA, Melo Júnior EB, Almeida PC, Macêdo SF, Silva ARV. Conhecimento, atitudes e práticas de medidas preventivas sobre pé diabético. Rev Gaúcha Enferm [Internet]. 2014 sep; [cited 12 feb 2018]; 35(3):36-42. Available from: http://www.scielo.br/scielo.php?pid=S1983-14472014000300036&script=sci_arttext&tlng=pt
  » http://www.scielo.br/scielo.php?pid=S1983-14472014000300036&script=sci_arttext&tlng=pt
• ⁹
  Oliveira AF, Marchi ACB, Leguisamo CP, Baldo GV, Wawginiak TA. Estimate of cost for diabetic foot treatment, how to prevent it and save funds. Ciênc Saúde Coletiva [Internet]. 2014 jun; [cited 20 feb 2018]; 19(6):1663-1671. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232014000601663
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232014000601663
• ¹⁰
  Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal Clinical Epidemiology [Internet]. 2010 jul; [cited 05 may 2018]; 63(7):737-45. Available from: https://www.ncbi.nlm.nih.gov/pubmed/20494804
  » https://www.ncbi.nlm.nih.gov/pubmed/20494804
• ¹¹
  Coluci MZO, Alexandre NMC, Milani D. Construction of measurement instruments in the area of health. Ciênc Saúde Coletiva [Internet]. 2015 mar; [cited 08 may 2018]; 20(3):925-936. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232015000300925
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232015000300925
• ¹²
  Apolinario D, Braga RCOP, Magaldi RM, Busse AL, Campora F, Brucki S, et al. Short assessment of health literacy for Portuguese-speaking adults. Rev Saúde Pública [Internet]. 2012 aug; [cited 16 apr 2018]; 46(4):702-11. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-89102012000400015
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-89102012000400015
• ¹³
  Marques SRL, Lemos SMA. Health literacy assessment instruments: literature review. Audiol Commun Res [Internet]. 2017; [cited 10 may 2018]; 22:e1757. Available from: http://www.scielo.br/scielo.php?pid=S2317-64312017000100501&script=sci_arttext&tlng=en
  » http://www.scielo.br/scielo.php?pid=S2317-64312017000100501&script=sci_arttext&tlng=en
• ¹⁴
  Eleutério TP, Pereira EJ, Farias PKS, Hott KPS, Paula FMT, Martins AMEBL. Elaboration and verification of the validity and reliability of a nutrition literacy instrument among people with diabetes. Cad Saúde Colet. [Internet]. 2018; [cited 27 mar 2019]; 26(3):298-307. Available from: http://www.scielo.br/pdf/cadsc/v26n3/1414-462X-cadsc-1414-462X201800030028.pdf
  » http://www.scielo.br/pdf/cadsc/v26n3/1414-462X-cadsc-1414-462X201800030028.pdf
• ¹⁵
  Sapnas KG, Zeller RA. Minimizing sample size when using exploratory factor analysis for measurement. J Nurs Meas [Internet]. 2002; [cited 20 apr 2018]; 10(2):135-54. Available from: https://www.ncbi.nlm.nih.gov/pubmed/12619534
  » https://www.ncbi.nlm.nih.gov/pubmed/12619534
• ¹⁶
  Hair Jr JF, Black WC, Babin B, Anderson RE, Tatham RL. Análise multivariada de dados. 6ª ed. Porto Alegre: Bookman Editora; 2009.
• ¹⁷
  Luiz RR, Magnanini MF. A lógica da determinação do tamanho da amostra em investigações epidemiológicas. Cad Saúde Coletiva [Internet]. 2000; [cited 23 jan 2018]; 8(2):9-28. Available from: http://www.cadernos.iesc.ufrj.br/cadernos/images/csc/2000_2/artigos/csc_v8n2_09-28.pdf
  » http://www.cadernos.iesc.ufrj.br/cadernos/images/csc/2000_2/artigos/csc_v8n2_09-28.pdf
• ¹⁸
  Bertolucci PHF, Brucki SMD, Campacci SR, Juliano Y. O mini-exame do estado mental em uma população geral: impacto da escolaridade. Arq Neuropsiquiatr [Internet]. 1994 mar; [cited 16 apr 2018]; 52(1):1-7. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0004-282X1994000100001&lng=en&nrm=iso&tlng=pt
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0004-282X1994000100001&lng=en&nrm=iso&tlng=pt
• ¹⁹
  Boell JEW, Ribeiro RM, Silva DMGV. Risk factors for developing diabetic foot. Rev Eletr Enf [Internet]. 2014 apr/jun; [cited 18 feb 2018]; 16(2):386-93. Available from: https://www.fen.ufg.br/revista/v16/n2/pdf/v16n2a15.pdf
  » https://www.fen.ufg.br/revista/v16/n2/pdf/v16n2a15.pdf
• ²⁰
  Rezende-Neta DS, Silva ARV, Silva GRF. Adherence to foot self-care in diabetes mellitus patients. Rev Bras Enferm [Internet]. 2015 jan/feb; [cited 24 mar 2018]; 68(1):111-6. Available from: https://www.ncbi.nlm.nih.gov/pubmed/25946502
  » https://www.ncbi.nlm.nih.gov/pubmed/25946502
• ²¹
  Silva LWS, Silva JS, Squarcini CFR, Souza FG, Ribeiro VS, Gonçalves DF. Health Promotion of people with diabetes mellitus regarding diabetic foot care. Ciencia y Enfermaria. 2016;22(2):103-16.
• ²²
  Lima IG, Costa JFL, Oliveira AF, Borges Junior JN, Peixoto AS, Pancien MS, et al. Educating to prevent: the importance of information in diabetic foot care. Revista Conexão UEPG. 2017;13(1):186-195.
• ²³
  Salomé GM, Almeida SA, Mendes B, Carvalho MRF, Bueno JC, Massahud MR, Ferreira LM. Association of Sociodemographic Factors with Spirituality and Hope in Patients with Diabetic Foot Ulcers. Adv Skin Wound Care [Internet]. 2017 jan; [cited 08 aug 2018]; 30(1):34-9. Available from: https://www.ncbi.nlm.nih.gov/pubmed/27984272
  » https://www.ncbi.nlm.nih.gov/pubmed/27984272
• ²⁴
  Davis LL. Instrument review: getting the most from a panel of experts. Appl Nurs Res [Internet]. 1992 nov; [cited 02 apr 2018]; 5(4):194-7. Available from: https://www.sciencedirect.com/science/article/pii/S0897189705800084
  » https://www.sciencedirect.com/science/article/pii/S0897189705800084
• ²⁵
  Grant JS, Davis LL. Selection and us of content experts for instrument development. Res Nurs Health [Internet]. 1997 jun; [cited 04 may 2018]; 20(3):269-74. Available from: https://www.ncbi.nlm.nih.gov/pubmed/9179180
  » https://www.ncbi.nlm.nih.gov/pubmed/9179180
• ²⁶
  Pedreira RBS, Rocha SV, Santos CA, Vasconcelos LRC, Reis MC. Validade de conteúdo do Instrumento de Avaliação da Saúde do Idoso. Einstein [Internet]. 2016 jun; [cited 10 may 2018]; 14(2):158-77. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1679-45082016000200009&lng=en&nrm=iso&tlng=pt
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1679-45082016000200009&lng=en&nrm=iso&tlng=pt
• ²⁷
  Damásio BF. Uses of exploratory factorial analysis in psychology. Avaliação Psicológica. 2012;11(2):213-28.
• ²⁸
  Valim MD, Marziale MHP, Hayashida M, Rocha FLR, Santos JLF. Validity and reliability of the Questionnaire for Compliance with Standard Precaution. Rev Saúde Pública [Internet]. 2015 dec; [cited 07 feb 2018]; 49:87. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4687826/
  » https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4687826/
• ²⁹
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics [Internet]. 1977 mar; [cited 07 feb 2018]; 33(1):159-74. Available from: https://www.ncbi.nlm.nih.gov/pubmed/843571
  » https://www.ncbi.nlm.nih.gov/pubmed/843571
• ³⁰
  Souza AC, Alexandre NMC, Guirardello EB. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol Serv Saude [Internet]. 2017 jul/sep; [cited 17 apr 2018]; 26(3):649-59. Available from: https://www.ncbi.nlm.nih.gov/pubmed/28977189
  » https://www.ncbi.nlm.nih.gov/pubmed/28977189
• ³¹
  Cronbach JL. My current t procedures. Educational and Psychological Measurement. 2004;64(3):391-418.
• ³²
  Davis TC, Long SW, Jackson RH, Mayeaux EJ, George RB, Murphy PW, et al. Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med [Internet]. 1993 jun; [cited 12 may 2018]; 25:391-5. Available from: https://www.ncbi.nlm.nih.gov/pubmed/8349060
  » https://www.ncbi.nlm.nih.gov/pubmed/8349060
• ³³
  Parker RM, Baker DW, Williams MV, Nurss JR. The test of functional health literacy in adults: a new instrument for measuring patient's literacy skills. J Gen Intern Med [Internet]. 1995 oct; [cited 07 may 2018]; 10:537-41. Available from: https://www.ncbi.nlm.nih.gov/pubmed/8576769
  » https://www.ncbi.nlm.nih.gov/pubmed/8576769
• ³⁴
  Caires IS, Goldberger BU, Colares MFA, Gale R, Grant J, Troncon LEA. Tradução, Adaptação, Validação e Avaliação para Uso no Brasil de um Instrumento Britânico de Auxílio à Escolha da Especialidade Médica. Rev Bras Educ Med [Internet]. 2017 jul/sep; [cited 28 may 2018]; 41(4):540-50. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0100-55022017000300379&lng=en&nrm=iso&tlng=pt
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0100-55022017000300379&lng=en&nrm=iso&tlng=pt
• ³⁵
  Pasquali L. Psicometria. Rev Esc Enferm USP. 2009;43(esp):992-9.
• ³⁶
  Marques SRL, Escarce AG, Lemos SMA. Health literacy and self-rated health in adults primary care patients. CoDAS [Internet]. 2018; [cited 08 aug 2018]; 30(2):e20170127. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S2317-17822018000200307&lng=en
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S2317-17822018000200307&lng=en
• ³⁷
  Jović-Vraneš A, Bjegović-Mikanović V, Marinković J, Vuković D. Evaluation of a health literacy screening tool in primary care patients: evidence from Serbia. Health Promot Int [Internet]. 2014 dec; [cited 06 mar 2018]; 29(4):601-7. Available from: https://www.ncbi.nlm.nih.gov/pubmed/23445940
  » https://www.ncbi.nlm.nih.gov/pubmed/23445940
• ³⁸
  Rocha PC, Lemos SMA. Conceptual aspects and factors associated with Functional Health Literacy: a literary review. Rev CEFAC [Internet]. 2016 feb; [cited 08 aug 2018]; 18(1):214-25. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1516-18462016000100214&lng=en
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1516-18462016000100214&lng=en
• ³⁹
  Marques SRL, Lemos SMA. Health literacy and associated factors in adults primary care users. Trab Educ Saúde [Internet]. 2018 mar/aug; [cited 08 aug 2018]; 16(2):535-59. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1981-77462018000200535&lng=pt&nrm=iso
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1981-77462018000200535&lng=pt&nrm=iso
• ⁴⁰
  Sampaio HAC, Carioca AAF, Sabry MOD, Santos PM, Coelho MAM, Passami MPB. Health literacy in type 2 diabetics: associated factors and glycemic control. Ciênc Saúde Coletiva [Internet]. 2015 mar; [cited 25 apr 2018]; 20(3):865-74. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232015000300865
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232015000300865
• ⁴¹
  Pedersen AB, Mikkelsen EM, Cronin-Fenton D, Kristensen NR, Pham TM, Pedersen L, et al. Missing data and multiple imputation in clinical epidemiological research. Clinical Epidemiology [Internet]. 2017 mar; [cited 17 may 2018]; 9:157-166. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5358992/
  » https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5358992/
• ⁴²
  Donders ART, Heijden GJMG, Stijnen T, Moons KGM. Review: A gentle introduction to imputation of missing values. J Clin Epidemiol [Internet]. 2006 oct; [cited 17 may 2018]; 59:1087-91. Available from: https://www.ncbi.nlm.nih.gov/pubmed/16980149
  » https://www.ncbi.nlm.nih.gov/pubmed/16980149
• ⁴³
  Rodrigues FFL, Santos MA, Teixeira CRS, Gonela JT, Zanetti ML. Relationship between knowledge, attitude, education and duration of disease in individuals with diabetes mellitus. Acta Paul Enferm [Internet]. 2012; [cited 21 may 2018]; 25(2):284-90. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0103-21002012000200020
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0103-21002012000200020
• ⁴⁴
  Santos MIPO, Portella MR. Conditions of functional health literacy of an elderly diabetics group. Rev Bras Enferm [Internet]. 2016 feb; [cited 20 may 2018]; 69(1):144-152. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672016000100156
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-71672016000100156
• ⁴⁵
  Assunção SC, Fonseca AP, Silveira MF, Caldeira AP, Pinho L. Knowledge and attitude of patients with diabetes mellitus in Primary Health Care. Esc Anna Nery [Internet]. 2017; [cited 28 may 2018]; 21(4):e20170208. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1414-81452017000400238&lng=en&nrm=iso&tlng=pt
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1414-81452017000400238&lng=en&nrm=iso&tlng=pt
• ⁴⁶
  Shin WK, Braun TM, Inglehart MR. Parents' dental anxiety and oral health literacy: effects on parents' and children's oral health-related experiences. J Public Health Dent [Internet]. 2014; [cited 28 may 2018]; 74(3):195-201. Available from: https://www.ncbi.nlm.nih.gov/pubmed/24329522
  » https://www.ncbi.nlm.nih.gov/pubmed/24329522

Publication Dates

• Publication in this collection
  18 July 2019
• Date of issue
  2019

History

• Received
  11 Nov 2018
• Accepted
  15 Apr 2019