SciELO - Scientific Electronic Library Online

 
vol.71 suppl.4Enseñanza del control de infecciones en la graduación en salud: la opinión de expertosCalidad de vida y actitudes alimentarias de estudiantes universitarios del área de la salud índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

Compartir


Revista Brasileira de Enfermagem

versión impresa ISSN 0034-7167versión On-line ISSN 1984-0446

Rev. Bras. Enferm. vol.71  supl.4 Brasília  2018

http://dx.doi.org/10.1590/0034-7167-2017-0648 

RESEARCH

Construction and validation of an Educational Content Validation Instrument in Health

Construcción y validación de un Instrumento de Validación de Contenido Educativo en Salud

Sarah de Sá LeiteI 

Aline Cruz Esmeraldo ÁfioI 

Luciana Vieira de CarvalhoI 

Jacqueline Mota da SilvaI 

Paulo César de AlmeidaII 

Lorita Marlena Freitag PagliucaI 

IUniversidade Federal do Ceará. Fortaleza, Ceará, Brazil.

IIUniversidade Estadual do Ceará, Nursing Department. Fortaleza, Ceará, Brazil.

ABSTRACT

Objective:

to construct and validate the Educational Content Validation Instrument in Health.

Method:

methodological study that includes the establishment of the conceptual structure; definition of objectives and population; construction of items and response scale; selection and arrangement of items; instrument structuring; opinion of experts; pre-test and content validation.

Results:

an instrument with 15 items was constructed and, after expert evaluation, eighteen items were obtained, divided into three domains: objectives (four), structure/presentation (nine), and relevance (two). Six items were modified since they presented a percentage of agreement below 0.8. Items of the total instrument presented good internal consistency (0.877) regarding domains.

Conclusion:

an Educational Content Validation Instrument in Health was elaborated and validated, presenting good reliability, and may contribute to the practice of researchers and health professionals in the development of educational content.

Descriptors: Methodological Research in Nursing; Health Education; Data Collection; Validation Studies; Nursing

RESUMEN

Objetivo:

construir y validar el Instrumento de Validación de Contenido Educativo en Salud.

Método:

estudio metodológico contemplando el establecimiento de la estructura conceptual; definición de los objetivos y población; construcción de los ítems y escala de respuesta; selección y organización de los ítems; estructura del instrumento; opinión de expertos; pre-test y validación de contenido.

Resultados:

se construyó un instrumento con quince ítems y después de la evaluación de expertos, se obtuvieron dieciocho ítems, divididos en tres áreas: objetivos (cuatro), estructura/presentación (nueve) y relevancia (dos). Seis ítems fueron modificados, pues presentaron un porcentaje de concordancia inferior a un 0,8. Los ítems del instrumento total presentaron una buena consistencia interna (0,877) en cuanto a sus dominios.

Conclusión:

fue elaborado y validado el Instrumento de Validación de Contenido Educativo en Salud, que presentó buena confiabilidad, contribuyendo a la práctica de investigadores y profesionales de las áreas de salud en la elaboración de contenidos educativos.

Descriptores: Investigación Metodológica en Enfermería; Educación en Salud; Recolección de Datos; Estudios de Validación; Enfermería

INTRODUCTION

Materials with educational content are facilitators of the teaching-learning process, enabling the transfer of knowledge, through the individual’s participation and involvement, and the exchange of experiences conducive to the improvement of skills(1). Such materials have been widely used for health education, which is a vehicle of knowledge socialization for contributing to the improvement of living conditions and health of the population(2).

Educational materials should be properly prepared and evaluated before being used by the target population. One of the essential steps for developing an effective educational material is the content validation, a process that evaluates its representativeness in properly addressing the universe with which it intends to deal and aims to measure or attend to the absence of unnecessary elements(3).

Currently, measuring instruments have been used to aid the content validation, constituting tools that measure indicators and attribute numeric values to abstract concepts, which can be observable and measurable. Thus, they contribute to enhancing the health praxis(4).

To develop tools that validate educational contents, an essential step is defining the construct to be investigated(5). Among the aspects evaluated by such instruments, issues involving promotion, prevention, and recovery of health are frequent, such as quality of life, vitality and limitations, adherence to treatment, and emotional and psychosocial factors(6).

However, there are recurring problems concerning the validation of instruments related to educational contents in health. It should be noted that inappropriate validation forms, without rigorous methodological criteria, the absence of instruments validated by capable professionals, as well as the fragmentation of educational instruments for health in topics or areas of interest, contribute to this process not always be effective, hindering the improvement and dissemination of knowledge about the phenomenon under study(7). This may result in the provision of educational materials with mistaken technical and didactic-pedagogical aspects. Thus, it is imperative that tools to validate educational material contents in the health areas can evaluate any themes, featuring proper reliability and validity. This will avoid unnecessary adaptation of instruments that validate educational contents for new research contexts, in addition to optimizing the time of researchers in this stage of data collection.

In the scientific literature, one can observe the absence of a universal validation instrument for contents with educational purposes. Given this context, the construction and validation of a reliable instrument through a scientific basis becomes necessary to validate several contents in health.

OBJECTIVE

To construct and validate the Educational Content Validation Instrument in Health (ECVIH), to be used by researchers and health professionals in the orientation and development of educational content.

METHOD

Ethical aspects

The study development complied with national and international ethical standards of research with human beings. This research was approved by the Research Ethics Committee of the Federal University of Ceará (UFC). Experts who participated in this study were clarified as to the purposes of the research and the nature of data collection. Those who agreed in participate signed the Informed Consent Form (ICF).

Study design, location, and period

This is a methodological study that includes the establishment of the conceptual structure; definition of objectives and population; construction of items and response scale; selection and arrangement of items; instrument structuring; opinion of experts; pre-test, and content validation(4).

For the construction of the instrument, a narrative review(8) in the databases PubMed, Scopus, and Cinahl, and a theoretical framework based on the principles of the instructional design of Filatro(9) was used.

The instrument construction and validation steps were carried out from March to December 2016. The sixth and seventh step, referring to the experts’ opinion and to pre-testing, were performed in person in the Laboratory of Communication and Health (Labcom_Saúde) of UFC. The last step, content validation, occurred virtually.

Population or sample, inclusion and exclusion criteria

A total of 27 experts participated in the study. Were selected, by trial, the experts who met the following inclusion criteria: Ph.D. and/or Master’s Degree in Health areas, professional experience (clinical, teaching, or research), and an article published in indexed journals in the study area of interest. Experts who have not responded to the invitation to participated in the study were excluded.

Study protocol

The process of instrument construction, on the first step, deals with the theoretical foundation of what will be studied for defining construct operations and dimensionality. Such step was based on the narrative review(8) and in the principles of instructional design, of Filatro(9).

The narrative review occurred in the databases PubMed, Scopus, and Cinahl, using the controlled keywords “Validation Studies” and “Teaching Materials”, and the non-controlled “Measuring Instruments”. The following crossings were performed: “Validation Studies” and “Teaching Materials”; “Validation Studies” and “Measuring Instruments”; “Teaching Materials” and “Measuring Instruments”. This step was based on the guiding question: which evaluation criteria were used to validate educational contents?

As inclusion criteria, we defined: research available in full and with free access in the selected databases, in English, Portuguese, or Spanish languages, which used instruments to evaluate the content in educational materials and were published from 2006 to 2016. Editorials, letters to the editor, reflective studies, experience reports, annals of scientific events (abstracts), and duplicate publications were excluded. Thus, 1,600 articles were found, and fifty were included in the research. However, after reading the full texts, 36 articles were excluded, totaling fourteen analyzed articles.

The examined literature showed that items of instruments used to evaluate educational material in health focus three groups of content-related factors: objectives, structure and presentation, and relevance. Items belonging to the group “objects” were related to purpose, goals, or target of the use of educational materials. The group “structure and presentation” contained information concerning the general organization, structure, strategy, consistency, and sufficiency of the presentations. In “relevance”, the items assessed the significance degree of the educational content and its ability to cause impact, motivation and/or interest.

For the theoretical support of the evaluated instruments, it was also used a theoretical reference based on the principles of instructional design, which is defined as the systematic actions of planning and construction of teaching methods and/or materials that can be used for classroom teaching, blended education, or distance learning. According to these principles, educational materials should provide interactivity, be attractive and written in an appropriate language to the target audience, promote relevant and contextualized activities, allow the exchange of experiences, and present quality information(9). In this study, the items of the ECVIH sought to cover such instructions, which are regarded as essential for the development of materials for educational purposes.

Then, goals and population were defined. The ECVIH aims to provide a scientific basis for the content validation of educational materials in health. It is destined to higher-level health professionals who wish to construct and validate educational contents to any target audience.

Instrument construction refers to the preparation or selection of its content according to the construct operational definitions. The conceptual structure step allowed the construction of nineteen items, with response options by a Likert-type scale, being 0 = disagree, 1 = partially agree, and 2 = totally agree. Following, items were selected and organized to evaluate the relevance of the drafting, adopting the behavioral criterion, in addition to objectivity, simplicity, clarity, accuracy, validity, relevance, and interpretability(10). After that, eighteen items remained.

Subsequently, the instrument was structured. This step aimed at consolidating the previous ones, namely, organizing the content to obtain a logical presentation order and thus establish the general format. Therefore, the ECVIH was structure by subdividing it into objectives, structure/presentation, and relevance. The last three steps were held with experts in the area of interest in measuring instrument, educational material, and health education.

Five experts in health, education, and statistics participated in the step of opinions and validation of the constructed items, to expand the diversity in the studied area of knowledge. All of them were Ph.Ds., had a scientific production and more than 10 years of experience in the area of interest of this study. The nominal group technique was adopted, being interactive, face-to-face, simultaneous, and held in group consensus(11).

Eleven experts participated in the instrument pre-testing, all of them having from three to five years of experience in the development of methodological studies in the area of health. Data collection happened only once, in and individual encounter, and came because of the discussion on instrument items and associated appendices.

The validation process counted with eleven experts – nine nurses and two pedagogues. The mean age was 38.9 ± 10,7 years. As for professional qualification, nine were Ph.Ds. and two had Master’s degree. We emphasize that tens experts were professors in Brazilian public universities (UFC, UECE, UNILAB, UNICAMP, and UFPI) and all had more than five years of scientific production in the study area. This process was made possible by electronic mail, via invitation letter and request for assessing the ECVIH regarding clarity, pertinence, relevance, and dimension in the writing of each item.

Analysis of results and statistics

Data were analyzed by the intraclass correlation coefficient (ICC), being regarded as great reliability (ICC > 0.9), good reliability (0.7 ≤ ICC ≤ 0.8), and poor reliability for values below 0.6(12). The significance level adopted was 95%. An agreement percentage of 80% was considered as a criterion for deciding on the pertinence of the instrument items, or its modification.

RESULTS

The initially constructed instrument comprised eighteen items and, after the five-expert consensus, fifteen remained (Chart 1). It should be noted that, after pre-testing, there was no modification in the items devised. Eleven experts have then performed the content validation process for the instrument items, following the criteria: clarity, practical pertinence, theoretical relevance, and theoretical dimension (Table 1). Items that did not reach an 80% agreement percentage were reformulated.

Table 1 Distribution of items of the Educational Content Validation Instrument in Health (ECVIH) according to the percentage of expert agreement, Fortaleza, Ceará, Brazil, 2017 

Items CL* PP* TR* TD*
(%) (%) (%) (%)
Objectives
Contemplates the proposed theme 100.0 100.0 100.00 100.00
Suits the teaching-learning process 90.9 90.9 90.9 90.9
Clarifies doubts on the addressed subject 90.9 81.8 90.9 81.8
Stimulates the learning about the addressed subject 81.8 100.0 100.0 81.8
Structure/presentation
Proper language 54.5 90.9 90.9 100.0
Interactive language 72.7 100.0 100.0 100.0
Correct information 72.7 90.9 100.0 100.0
Objective information 81.8 100.0 100.0 100.0
Understandable information 54.5 81.8 90.9 90.9
Logical sequence of ideas 81.8 90.9 100.0 100.0
Current thematic content 72.7 100.0 100.0 90.9
Appropriate text size 90.9 100.0 100.0 100.0
Content free of discrimination or prejudice 90.9 90.9 90.9 90.9
Relevance
Provides reflection on the subject 81.8 100.0 90.9 81.8
Encourages behavior change 81.8 90.9 81.8 72.7

Note:

*CL: clarity of language; PP: practical pertinence; TR: theoretical relevance; TD: theoretical dimension.

Chart 1 Original items, excluded, modified and/or maintained in the Educational Content Validation Instrument in Health (ECVIH) according to the experts' evaluations, Fortaleza, Ceará, Brazil, 2017 

ECVIH* Itens Evaluation
1. Contemplates the universe of the proposed theme Modified
2. Has a direct relationship with the target audience Excluded
3. Suits the teaching-learning process Maintained
4. Clarifies doubts on the addressed subject Maintained
5. Stimulates learning about the addressed subject Maintained
6. Proper language Maintained
7. Interactive language Maintained
8. Correct information Maintained
9. Clear information Excluded
10. Objective information Maintained
11. Understandable information Maintained
12. Logical sequence of ideas Maintained
13. Current content Maintained
14. Appropriate text size Maintained
15. Free of discrimination or prejudice Maintained
16. Provides reflection on the subject Maintained
17. Encourages behavior change Maintained
18. Stresses the importance of the content Excluded

Note:

*Educational Content Validation Instrument in Health.

In the reliability assessment, the ECVIH presented a satisfactory total internal consistency (ICC > 0.8) and a significant p-value (p < 0.05), similarly to the domains “structure/presentation” and “relevance”. However, in the domain “objectives”, a weak internal consistency was identified (ICC = 0.432) and p-value > 0.05, i.e. indicating the need for adjustments to increase domain reliability (Table 2).

Table 2 Analysis of the internal consistency of the domains analyzed in the Educational Content Validation Instrument in Health (ECVIH), Fortaleza, Ceará, Brazil, 2017 

Domains ICC* IC95% p value
Objectives 0.432 [-0.212; 0.819] 0.072
Structure and presentation 0.857 [0.702; 0.954] <0.0001
Relevance 0.853 [0.672; 0.954] <0.001
ECVIH** Total 0.877 [0.746; 0.960] <0.0001

Note:

*ICC: Intraclass Correlation Coefficient;

**ECVIH: Educational Content Validation Instrument in Health

The final configuration of the ECVIH has 18 items, divided into three areas: objectives, structure/presentation, and relevance. The total score of the instrument is calculated through the sum of all domains (Chart 2).

Chart 2 Educational Content Validation Instrument in Health, Fortaleza, Ceará, Brazil, 2017 

OBJECTIVES: purposes, goals, or targets 0 1 2
1. Contemplates the proposed theme
2. Suits the teaching-learning process
3. Clarifies doubts on the addressed theme
4. Provides reflection on the theme
5. Encourages behavior change
STRUCTURE/PRESENTATION: organization, structure, strategy, consistency, and sufficiency 0 1 2
6. Language appropriate to the target audience
7. Language appropriate to the educational material
8. Interactive language, enabling active involvement in the educational process
9. Correct information
10. Objective information
11. Enlightening information
12. Necessary information
13. Logical sequence of ideas
14. Current theme
15. Appropriate text size
RELEVANCE: significance, impact, motivation, and interest 0 1 2
16. Encourages learning
17. Contributes to knowledge in the area
18. Arouses interest in the theme

Note: Items valuation: 0 disagree; 1 partially agree; 2 totally agree.

DISCUSSION

Developing and validating the ECVIH contributes to the clinical and scientific praxis because this instrument is an innovative tool to be used to validate educational content available in materials such as videos, albums, booklets, games, websites, and software. Moreover, considering that it does not specify information about the theme, target audience, and application circumstances, the instrument can support educational activities in health.

The use of the nominal group technique was advantageous for the ECVIH construction since the consensus on the maintenance, modification, or exclusion of items much contributed for the construction of a more reliable instrument due to the evaluators’ expertise(11).

The prior use of the instrument in a small sample, characterizing a pre-test, allowed us to reach a preliminary version with more harmonic methodological choices.

For the ECVIH validation process, the experts evaluated three domains, namely: objectives, structure/presentation, and relevance. In the first domain, the modifications consisted in the inclusion of the items “provides reflection on the subject” and “encourages behavior changes”, as well as the offset of the item “stimulates the learning” to another domain. Although the five items belonging to this domain presented an agreement percentage of more than 80% among the experts in all criteria assessed, the researchers complied with the modifications suggested by the experts to improve the ECVIH items.

Educational materials evaluated by content instruments in the health area should aim at facilitating the work of health professionals in orienting and educating patients and family(13). Furthermore, they must aim at promoting healthy, ensuring the effectiveness of care for the target population. Therefore, the domain “objective” of an instrument is required for the understanding on the content to be studied.

In the domain “structure/presentation,” the items complied with the criteria of practical pertinence and theoretical relevance and dimension, with an agreement of more than 80% among experts, requiring adjustments in five items to make them more understandable since they featured low agreement on the criterion language clarity. The main modifications were in relation to the items writing, such as replacing “proper language” for “language appropriated to the target audience” and “interactive language” for “Interactive language, enabling active involvement in the educational process”. We also decided to standardize nomenclatures, such as “thematic content” and “theme”.

Paying attention to the evaluation of educational materials in health is essential, through specific content instruments, especially regarding a consistent, cohesive, organized, sufficient structure, and an appropriate language. Regarding cohesion and coherence, the content must remain focused on the proposed theme, and in its topics, paragraphs, or portions there should be a logical sequence of ideas, starting from general and impersonal issues until more delicate or specific subjects. Sufficiency is one of the aspects closely related to material validity since sufficient educational contents explain relevant topics of the proposed theme in a comprehensive way, in addition to preventing doubts of the reader(9,14).

Among the necessary aspects for the language of a good measuring instrument, objectivity and clarity are highlighted. Long and detailed sentences may lead to dispersion and make the reading tiresome, and an extensive instrument can compromise the willingness in replying it. Ambiguity caused by the use of vague terms is another potential problem related to the language, which may result in inadequate and inaccurate responses(15-16). To avoid such problems, one of the aspects this study has prioritized was the use of a clear and direct language. The ECVIH items are objective, comprising solely the words required for understanding what is being measured. The size of the instrument also contributes to its fast and simple filling since it consists of only eighteen items. We also emphasize that each item measures only one attribute. Therefore, the conciseness is considered the strong point of this instrument.

Regarding “relevance”, the experts suggested significant changes in the items of this domain, such as the exchange of some items with the dimension “objective”, in addition to the inclusion of the items “contributes to knowledge in the area” and “arouses interest in the topic”.

For the content evaluated by health areas instruments to be current and relevant, it is essential that the researcher is inserted in the social context of the target population, knowing its needs and circumstances. Theme and topics covered in educational contents should meet the needs of the participant, promoting empowerment on the addressed subject. A previous study on education through a simple educational brochure on low-carbohydrate and low-sugar diet shows the increased knowledge on the theme after using the resource, in addition to emphasizing the importance of educational materials with this purpose(17).

In the reliability assessment, the total ECVIH presented a good level of internal consistency, as well as the domains “structure/presentation” and “relevance”, and a weak level only for the domain “objective”. It is important to mention that, although the instrument present this weakness, the recommended modifications enabled the improvement of reliability in the domains “objective”, considering the experience of the experts in devising and validating the measuring instrument and in educational contents in the area of health.

Thus, the ECVIH points to several relevant aspects to educational content, which are essential to fulfill the purposes of this kind of material and provide education as efficiently as possible. It is therefore considered that the instrument was able to meet the purposes for which it was developed, being able to orientate the construction of educational contents in health, in different formats, since it was overall considered appropriate by the experts.

Study limitations

As a limitation, we emphasize that the instrument here constructed and validated aims solely to evaluate the written content of education materials, being excluded pictures, images, and drawings. It is understood that these items will be evaluated in the subsequent validation process of construct appearance.

Contributions to the field of nursing, health, or public policies

This study will contribute to the practice of researchers and health professionals in developing contents for educational materials that go beyond health and traditional education.

It is believed that the elaboration of a research instrument based on general standards may be the first step to enhance educational intervention in health, evaluating multi-professional phenomena in different health contexts.

CONCLUSION

This study constructed and validated the ECVIH, which presented a good reliability (ICC > 0.8) and value p < 0.05. The participation of specialists in the development and validation of this instrument was essential to avoid inaccurate results or biased measures that could lead to wrong conclusions. It should be noted that the evaluation of instruments in the field of health must be multi-professional, aiming to increase phenomenon objectivity.

Thus, the final configuration of the instrument has eighteen items, divided into three domains, with response options ranging from 0 to 2. We considered the ECVIH a valid and reliable alternative to evaluate educational contents in health.

It is expected that, in subsequent studies, the clinical validation will be performed, to evaluate the efficiency of the instrument here constructed and validated as a facilitator in obtaining data on educational materials in the health area.

FUNDING

The Ceará Foundation of Support to the Scientific and Technological Development allowed financial support to the research.

REFERENCES

1 Barros EJL, Santos SSC, Gomes GC, Erdmann AL. Gerontotecnologia educativa voltada ao idoso estomizado à luz da complexidade. Rev Gaúcha Enferm [Internet]. 2012 [cited 2016 Nov 28];33(2):95-101. Available from: http://www.scielo.br/pdf/rgenf/v33n2/14.pdfLinks ]

2 Salci MA, Maceno P, Rozza SG, Silva DMGV, Boehs AE, Heidemann ITSB. Health education and its theoretical perspectives: a few reflections. Texto Contexto Enferm[Internet]. 2013[cited 2017 Jan 12];22(1):224-30. Available from: http://www.scielo.br/pdf/tce/v22n1/27.pdfLinks ]

3 Tibúrcio MP, Melo GSM, Balduíno LSC, Freitas CCS, Costa IKF, Torres GV. Content validation of an instrument to assess the knowledge about the measurement of blood pressure. Rev Pesqui Cuid Fundam[Internet]. 2015[cited 2017 Jan 15];7(2):2475-85. Available from: http://www.seer.unirio.br/index.php/cuidadofundamental/article/view/3585/pdf_1578Links ]

4 Coluci MZO, Alexandre NMC, Milani D. Construção de instrumentos de medida na área da saúde. Ciênc Saúde Colet[Internet]. 2015[cited 2016 Nov 28];20(3):925-36. Available from: http://www.scielo.br/pdf/csc/v20n3/1413-8123-csc-20-03-00925.pdfLinks ]

5 Segabinazi JD, Zortea M, Zanon C, Bandeira DR, Giacomoni CH, Hutz CS. Escala de afetos positivos e negativos para adolescentes: adaptação, normatização e evidências de validade. Aval Psicol[Internet]. 2012[cited 2017 Jan 31];11(1):1-12. Available from: http://pepsic.bvsalud.org/pdf/avp/v11n1/v11n1a02.pdfLinks ]

6 Rattray J, Jones MC. Essential elements of questionnaire design and development. J Clin Nurs [Internet]. 2007[cited 2017 Jan 25];16(2):234-43. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2702.2006.01573.x/epdfLinks ]

7 Cano SJ, Hobart JC. The problem with health measurement. Patient Prefer Adher [Internet]. 2011 [cited 2017 Jan 31];5:279-90. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3140310/pdf/ppa-5-279.pdfLinks ]

8 Elias CSR, Silva LA, Martins MTSL, Ramos NAP, Souza MGG, Hipólito RL. When is the end? a narrative review on the termination of the school term for mentally disable students. Rev Eletr Saúde Mental Álcool Drog [Internet]. 2012[cited 2017 Jan 31];8(1):1-6. Available from: http://www.revistas.usp.br/smad/article/view/49594/53719Links ]

9 Filatro A. Design instrucional contextualizado: educação e tecnologia. 3ª ed. São Paulo: Senac; 2011. [ Links ]

10 Cardoso CS, Bandeira M, Ribeiro ALP, Oliveira GL, Caiaffa WT. Escalas de satisfação com o atendimento às doenças cardiovasculares: CARDIOSATIS usuário e equipe. Ciênc Saúde Colet[Internet]. 2011 [cited 2016 Oct 12];16(Supl-1):1401-07. Available from: http://www.scielo.br/pdf/csc/v16s1/a75v16s1.pdfLinks ]

11 McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm [Internet]. 2016 [cited 2017 Jan 31];38(3):655-62. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4909789/pdf/11096_2016_Article_257.pdfLinks ]

12 Pestana MH, Gageiro JN. Análise de dados para Ciências Sociais: a complementaridade do SPSS. 4ª ed. Lisboa: Edições Sílabo; 2005. [ Links ]

13 Áfio ACE, Balbino AC, Alves MDS, Carvalho LV, Santos MCL, Oliveira NR. Analysis of the concept of nursing educational technology applied to the patient. Rev Rene [Internet]. 2014 [cited 2016 Nov 28];15(1):158-65. Available from: http://www.redalyc.org/pdf/3240/324030684020_2.pdfLinks ]

14 Khurana S, Rao BK, Lewis LES, Bhat R, Purkayastha J, Kamath A, et al. Development and Validation of Educational Leaflet for Caregivers of Preterm Infants. J Clin Diagn Res[Internet]. 2016[cited 2017 Jan 31];10(7):1-4. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5020196/pdf/jcdr-10-YC01.pdfLinks ]

15 Keszei AP, Novak M, Streiner DL. Introduction to health measurement scales. J Psychosom Res [Internet]. 2010[cited 2017 Jan 31];68(4):319-23. Available from: http://www.jpsychores.com/article/S0022-3999(10)00011-5/pdfLinks ]

16 Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res [Internet]. 2010[cited 2016 Nov 28];19:539-49. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2852520/pdf/11136_2010_Article_9606.pdfLinks ]

17 An YM, Jun DW, Lee SM. Development and application of low-carbohydrates and low-simple sugar nutrition education materials for non-alcoholic fatty liver disease patients. Clin Nutr Res[Internet]. 2015[cited 2016 Nov 29];4(4):250-58. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4641987/pdf/cnr-4-250.pdfLinks ]

Recibido: 21 de Septiembre de 2017; Aprobado: 27 de Enero de 2018

CORRESPONDING AUTHOR: Sarah de Sá Leite E-mail: sarahsaleite@hotmail.com

Creative Commons License This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.