Content validation of an infant evaluation instrument

Abstract

Objective

To validate the content of an infant evaluation instrument based on the interventions proposed by the First Integral Health Week.

Methods

Methodological research developed in two phases: definition of the instrument variables and content validation. The application of the Delphi Technique was used with a minimum consensus level of 70%. To validate the internal consistency of the tool, the reliability estimator Cronbach’s alpha and the estimator greatest lower bound were applied.

Results

The instrument was considered appropriate in terms of general lay-out (90%), easy understanding (90 to 100%), feasibility for care practice (100%) and items addressed (100%).

Conclusion

The instrument developed demonstrated content validity and compatibility to assess the infant in the first week of life. Therefore, it can serve as a practical tool to qualify and guide the health interventions involving infants in the context of Primary Health Care.

Child health; Child health services; Maternal-child health services; Validation studies

Resumo

Objetivo

Validar o conteúdo de um instrumento de avaliação do recém-nascido baseado nas intervenções propostas pela Primeira Semana de Saúde Integral.

Métodos

Pesquisa metodólogica realizada em duas fases: definição das variaveis do instrumento e validação de conteúdo. Utilizou-se da aplicação da Técnica Delphi a um índice de 70% como nível mínimo de consenso. Para validar a consistência interna do instrumento, foram aplicados o estimador de confiabilidade alfa de Cronbach e o estimador greatest lower bound.

Resultados

O instrumento apresentou adequação quanto à aparência geral (90%), facilidade de entendimento (90 a 100%), viabilidade para prática assistencial (100%) e itens contemplados (100%).

Conclusão

O instrumento desenvolvido demonstrou validade de conteúdo e compatibilidade para avaliar o recém-nascido na primeira semana de vida. Poderá, portanto, constitui-se em ferramenta prática para qualificar e direcionar as intervenções em saúde realizadas aos recém-nascido no contexto na Atenção Primária à Saúde.

Saúde da criança; Serviços de saúde da criança; Serviços de saúde materno-infantil; Estudos de validação

Resumen

Objetivo

Validar el contenido de un instrumento para evaluación del recién nacido basado en las investigaciones propuestas por la Primera Semana de Salud Integral.

Métodos

Investigación metodológica, realizada en dos fases: definición de variables del instrumento y validación de contenido. Se aplicó la Técnica Delphi aun índice de 70% como nivel mínimo de consenso. Para validar la consistencia interna del instrumento, se utilizaron el estimador de confiabilidad Alfa de Cronbach y el greatest lower bound.

Resultados

El instrumento mostró adecuación respecto de apariencia general (90%), facilidad de comprensión (90 a 100%), viabilidad para la práctica asistencial (100%) e ítems contemplados (100%).

Conclusión

El instrumento desarrollado demostró validez de contenido y compatibilidad para evaluar al recién nacido en su primera semana de vida. Podrá, consecuentemente, constituirse en herramienta práctica para calificar y orientar las intervenciones en salud realizadas a recién nacidos en el ámbito de la Atención Primaria de Salud.

Salud del niño; Servicios de salud del niño; Servicios de salud materno-infantil; Estudios de validación

Introduction

Despite advances in child survival rates, global statistics show that about 15% of expected births will result in life-threatening complications during pregnancy, childbirth or postpartum, especially in the first week of life.^(1,2)

In recognition of the relevance of the issue of maternal and neonatal mortality to the present day, the United Nations has proposed as one of the goals of the Sustainable Development Goals to eliminate preventable deaths of newborns and children under five, aiming to reduce neonatal mortality to 12 per 1,000 live births by 2030 (UN, 2015).⁽³⁾

The child in the first week of life is more vulnerable to problems related to biological, socioeconomic and health care determinants, this being a period considered as adaptive in the extra-uterine environment, which makes maternal and child follow-up indispensable for health promotion.^(4-6) In this context, professional care for infants should cover actions taken to identify the difficulties the mother faces in the care for the newborn and in the evaluation to detect early signs of illness in order to accomplish comprehensive care actions.⁽⁷⁾

Brazil presents advances in the fight against child mortality, but this fact is still considered a great social problem. This situation has motivated the Ministry of Health (MS) to establish, implement and organize strategies for the qualification of Maternal and Child Health Care Networks by establishing health programs and policies that cover actions for health promotion, disease prevention and health surveillance.^(5,8)

Among the proposals developed by the Ministry of Health to qualify infant care, since 1996 the Integrated Management of Childhood Illness (IMCI) strategy is in place. This is a global action, initially articulated by the Pan American Health Organization (PAHO) in partnership with the United Nations Children’s Fund (UNICEF), with the objective of improving the care provided by health professionals, based on the definition of a set of criteria to systematically assess, classify and treat diseases prevalent in children under five. The IMCI proposes to systematically and integrally evaluate clinical signs predictive of health problems.^(9,10)

Proceeding with the creation of strategies aimed at reducing neonatal mortality, in 2004, the agenda of Commitments to Integral Health of the Child and Reduction of Infant Mortality was created, which consists of recommending directives, departing from care lines for all levels of care, especially in Primary Care. One of the initiatives that stand out in this scenario has been the line of care “Humanized and qualified care for the pregnant woman and newborn”. This line of care contains actions guided by the First Integral Health Week (PSSI), which designates priorities for health interventions in the first seven days of the infant’s life.⁽¹¹⁾

The actions advocated by the PSSI include the monitoring of growth and development, being considered the guiding axis of comprehensive child health care. It provides for intersectoral actions that are proven to be effective and necessary for infant health promotion, but there is no systematization in the evaluation process that allows all newborns to be evaluated considering the same sequence of items and criteria recommended in the agenda of Commitments to the Integral Health of the Child and Reduction of Infant Mortality for the PSSI.^(5,12)

Thus, the purpose of this study was to validate the content of an infant evaluation instrument based on the interventions proposed by the First Integral Health Week; in the framework of the evaluation and classification procedures for children from 0 to 2 months of age in the Neonatal IMCI Manual and in the Technical Prenatal and Postpartum Care Manual: qualified and humanized care.

Methods

A descriptive, methodological development study with a quantitative approach. Content validation was applied using the Delphi technique, performed in two stages:⁽¹³⁾

a)Development of an infant evaluation instrument in the PSSI: The instrument was developed based on the actions recommended in the “First Integral Health Week” care line of the Agenda for Commitments for Integral Health of the Child and Reduction of Infant Mortality, in the framework of the assessment and classification procedures for children aged 0 to 2 months of the Neonatal IMCI Manual. This framework of procedures makes it possible to assess and determine the presence of severe illness or localized infection in the child and in the Technical Prenatal and Postpartum Manual: qualified and humanized care.^(12-15)

The composition of the instrument was divided into four blocks: Block A - identification data of the mother, father, child and health professional, obstetric history of the postpartum woman, general information about prenatal, childbirth and birth care. Block B - checklist of signs of danger for the newborn, based on the evaluation and classification items of the child from 0 to 2 months, according to the Neonatal AIDPI Manual. Block C - includes items to evaluate the actions advocated in the First Integral Health Week. These are: neonatal clinical screening, child’s vaccination status, guidelines on breastfeeding, and neonatal and maternal hygiene care. Block D - evaluation of the mother’s general condition (postpartum), risk situations for mother-baby, appointment making.

b) Content validation of the infant evaluation instrument in the PSSI using the Delphi Technique: the 70% index was adopted as the minimum consensus level to be obtained by the experts in the validation of the instrument.⁽¹⁶⁾ The study was carried out in three phases. The first one is the content analysis phase of the instrument by expert judges, assigning scores (1 - cannot be evaluated; 2 - bad; 3 - good; 4- optimal) regarding aspects of general layout, easy understanding, feasibility of care practice and relevance of topics. The second phase corresponds to the correction and incorporation of the suggestions in the instrument according to the expert analysis, and the third phase refers to the statistical validation of the instrument content.

To select the experts, a search was undertaken in the curricula available in the Lattes Platform and in the organizational charts of the Ministry of Health, the municipal and state health departments to identify the professionals who were working in Child Health Management and the Stork Network.

Fourteen professionals were invited to serve as experts. Of these, ten accepted to take part in the research. The selected judges received an e-mail invitation letter explaining the purpose of the study and the criteria for their selection, with a deadline of up to 20 days to return the evaluated material. All judges received a Free and Informed Consent Form (TCLE), a questionnaire to characterize the judges, a questionnaire on the instrument to be analyzed and a form with instructions.

The ten judges returned all instruments within the established timeframe and a 95% consensus was obtained for all items by blocks of the infant evaluation instrument in the First Integral Health Week. Therefore, the instrument was validated in the first Delphi phase.

The data were collected from April to June 2014 and analyzed by means of descriptive statistics, measuring the inter-rater percentage of agreement for each item per block. To validate the internal consistency in the judges’ responses, two measures were used to estimate reliability, Cronbach’s alpha and the glb (greatest lower bound) estimator.⁽¹⁶⁾ The study received approval from the Ethics and Research Committee of the Federal University of Amazonas, CAAE 28809114.1.0000.5020.

Results

Concerning the sociodemographic characteristics of the judges participating in the research, of the ten judges, nine were women, aged between 33 and 53 years, length of experience in neonatology and/or pediatrics between 05 and 26 years of work, lato sensu graduate (specialist) (7) and stricto sensu Master’s (2) and doctorate (1) degrees. In relation to the activities in the area of child health, four judges worked in primary health care, four in the management and planning of hospitalized child care with expertise in validation of educational technologies and one in higher education teaching.

There was a consensus of 95% among the judges regarding the items evaluated in Phase I of the Delphi Technique, which determined, in this phase, the conclusion of the instrument validation. For the analysis, the ten questionnaires returned were considered.

Considering the good and excellent concepts demonstrated, the consensus among the judges amounted to 90% in Block A, referring to the items general layout and easy understanding. In Block B (assessment of signs of danger in the newborn), there was unanimous agreement (100%) among the judges, thus achieving content validity. The evaluation of Block C pointed out the concepts good and excellent, with a consensus of 90 to 100% among the judges regarding the evaluated items. As to the items proposed in Block D of the instrument, the concepts good and excellent indicated 100% agreement among the judges.

The descriptive statistical analysis, calculating the central trend and variance measures among the judges’ scores, confirmed the consensus on the evaluated items (Table 1).

Table 1
Central trend and variance measures in expert judgment scores for content of infant evaluation instrument in First Integral Health Week - block A, B, C and D

Block	Mean	Median	SD	Max.	Min.
Block A
1. General layout	3.5	4.0	0.707	4	2
2. Easy understanding	3.7	4.0	0.675	4	2
3. Feasibility of care practice	3.8	4.0	0.422	4	3
4. Items covered	3.5	3.5	0527	4	3
Block B
5. General layout	3.9	4.0	0.316	4	3
6. Easy understanding	3.7	4.0	0.483	4	3
7. Feasibility of care practice	3.9	4.0	0.316	4	3
8. Items covered	3.8	4.0	0.422	4	3
Block C
9. General layout	3.7	4.0	0.675	4	2
10. Easy understanding	3.8	4.0	0.632	4	2
11. Feasibility of care practice	4.0	4.0	0.000	4	4
12. Items covered	3.8	4.0	0.632	4	2
Block D
13. General layout	3.9	4.0	0.316	4	3
14. Easy understanding	3.9	4.0	0.316	4	3
15. Feasibility of care practice	4.0	4.0	0.000	4	4
16. Items covered	4.0	4.0	0.000	4	4

• SD – Standard Deviation; Max. – Maximum; Min. – Minimum

According to table 1, analyzing the four items the judges evaluated, it was observed that, for Blocks A, B, C and D, the average varied between (3.5 and 3.8), (3.7 and 3.9), (3.7 to 4.0) and (3.9 to 4.0) respectively, close to the maximum score of the variation evidencing the consensus on the scores between good and excellent. In Block C, in the item feasibility for care practice, the maximum score was reached, corroborating the agreement among the judges in relation to this block of the instrument. The mean and median were close, which demonstrated symmetry in the data; the standard deviation evidenced that the degree of variability among the scores was minimal, confirming the consensus among the judges in relation to Blocks A, B, C and D of the instrument.

In relation to the modifications the judges suggested, concerning the items covered, two judges recommended suppressing the information number of the Local Information Manager (GIL), as it was not a universal information system, and this field was changed to National Health Card. In the same item, a judge proposed including a box for post-natal problems and two other judges suggested including the box type of delivery (normal, cesarean and forceps), suggestions incorporated into the instrument.

In Block C, one of the judges pointed out two suggestions, which were considered relevant and included in the instrument: complementing the inquiry about breastfeeding, including room to assess whether the newborn was in Exclusive Breastfeeding or in mixed breastfeeding; and the other was to insert a box to evaluate breast problems, in order to permit the evaluation of the woman’s breasts.

In Block D, in the item called Situations of risk for the baby and for the mother, in which the name Mother with hearing, visual and/or mental impairment appeared, a judge suggested changing the name, following current legislation, to Mother with physical, sensory (auditory and visual) and mental impairment.

Regarding the layout of the instrument, two judges suggested modifications in Block A and one judge considered that the layout of Block C was bad. The suggested modifications were accepted and related to increasing the font size and spacing to fill out the answers in order to facilitate reading and understanding of the instrument content. Regarding the pertinence of the instrument, Blocks A and C reached a 90% consensus and Blocks B and D 100% agreement among the judges.

Before verifying the internal consistency of the instrument, it was verified that the aspects were measuring the same variable. High correlations were found between items A1, A2, C1, C2, D1, D2, while the items in Block B were correlated with themselves. The items C3, D3, D4 are not present because they presented a standard deviation equal to zero (Figure 1). Thus, the aspects of General Layout and Easy Understanding, for Blocks A, C and D, measuring the same variable, were considered as a single aspect. As Block B dealt with signs of danger, with distinct aspects compared to the other three blocks, it can be affirmed that the items in this block are measuring the same variable on the quality of the items signalling danger.

Figure 1
Inter-item correlations. Items C3, D3 and D4 are not present due to lack of variation

In order to evaluate the internal consistency, the following dimensions were assumed: Dimension 1: general layout and easy understanding (items A1, A2, C1, C2, D1, D2); Dimension 2: questions on signs of danger (items B1, B2, B3, B4); Dimension 3: this dimension contains items A3, A4 and C4, although these items are related to the practical feasibility and the items addressed in the instrument, this does not disclose the existence of evidence that the items constitute a dimension but, for the sake of consistency calculations, they were aggregated into the dimension.

The internal consistency of the instrument is related to the ability of items of the same dimension to be consistent with what the dimension is actually intended to measure. For items in the same dimension, the internal consistency was measured through the reliability, which is the ratio between the item variance and the construct variance.

Before calculating the Cronbach’s alpha and the glb estimator, it was important to note that there were perfect correlations between the following pairs of items (Figure 1): (A2, C1), (C2, D1), (C2, D2), (D1, D2) and (B1, B3). The glb estimator cannot be calculated in the presence of perfect correlations. In addition, it is emphasized that Cronbach’s alpha tends to be high if there are measures with perfect correlations, which can generate an impossible estimate for reliability. Therefore, to measure the reliability of dimension 1, only items A1, A2 and C2 were used and, for the reliability of dimension 2, items B1, B2 and B4 were used.

The estimates of the lower limits for reliability using Cronbach’s alpha and glb were as follows: Dimension 1 - general layout and easy understanding: Cronbach’s alpha (0.9281) and glb (0.9302); Dimension 2 - signs of danger: Cronbach’s alpha (0.8365) and glb (0.8678); Dimension 3 - practical feasibility and items covered: Cronbach’s alpha (0.7404) and glb (0.7339).

Although there is no universal scale for reliability, it is common to regard values above 0.7 as acceptable, which turned the experts’ judgments consistent for the three dimensions.

Discussion

Considering the importance of the birth period for the mother and the newborn, the post-2015 global health agenda establishes that the survival of the newborn should become a global priority, with each country moving to promote best practices in special neonatal care, especially in the detection of signs of danger.^(17,18)

The clinical practice extended with the knowledge, feelings and experiences of the mother and family helps the health professionals in the diagnosis, planning of care and appropriate treatment. It should therefore be an action of cooperation and articulation among professional, mother and family. Hence the importance of the construction of an instrument to systemize care during this period.⁽¹⁹⁾

The indices obtained in the content validation process of the instrument studied indicated high reliability. The analysis by the expert judges contributed to the construction of the material insofar as they suggested changes in terminology and inclusion of evaluation items. All suggestions have been incorporated into the content, so that they will enhance the successful application of the instrument.

The preparation of an instrument for infant evaluation in the First Integral Health Week can collaborate in the qualification of perinatal care at all care levels, to the extent that we recognize situations of risk and provide appropriate and problem-solving care for the development of positive responses to the survival and quality of life of newborn infants, with the effective participation of the multiprofessional health team that works through neonatal clinical screening.^(20,21)

Despite the proven rigor in validating the content of the instrument, it is necessary to continue with the following phases, for the purpose of operational and measurement equivalence. To do so, it should be applied in practice, so that its efficiency can be verified.

Conclusion

The results obtained in the study of the reliability and validity of the infant evaluation instrument in the First Integral Health Week indicated acceptable psychometric properties for its use in public health services for child care in the first week of life. The judges’ consensus provided evidence for the construct (instrument) and content validation, including the items they recommended. The concurrent validity of each item, separately, and the instrument globally, were measured with a significant outcome, following the methodological rigor of the Delphi technique. Finally, future studies are suggested to verify the applicability of the infant evaluation instrument in the First Integral Health Week, aiming to contribute to the improvement of newborn care in Primary Health Care, with its publication through electronic information technology.

Referências

• ¹
  Otolorin E, Gomez P, Currie S, Thapa K, Dao B. Essential basic and emergency obstetric and newborn care: from education and training to service delivery and quality of care. International J Gynecol Obstet. 2015; 130(2):46-53.
• ²
  Lassi AS, Salam RA, Das JK, Bhutta ZA. Essential interventions for maternal, newborn and child health: background and methodology. Reprod Health. 2014; 11(Supll1):1.
• ³
  Organização das Nações Unidas (ONU). Guia sobre desenvolvimento sustentável- 17 objetivos para transformar o nosso mundo. Genebra; ONU: 2015
• ⁴
  Teixeira GA, Costa FM, Mata MS, Carvalho JB, Souza NL, Silva RA. Fatores de risco para a mortalidade neonatal na primeira semana de vida. Fundam Care Online. 2016; 8(1):e4036-46.
• ⁵
  Gaiva MA, Dias NS, Siqueira VC. Atenção ao neonato na estratégia saúde da família: avanços e desafios para a atenção integral. Cogitare Enferm. 2012; 17(4):730-7.
• ⁶
  Castro AC, Duarte ED, Diniz IA. Intervenção do enfermeiro às crianças atendidas no ambulatório de seguimento do recém-nascido de risco. Rev Enferm Centro-Oeste Mineiro. 2017; 7(1159):1-8.
• ⁷
  Reichert AP, Guedes AT, Pereira VE, Cruz TM, Santos NC, Collet N. Primeira semana saúde integral: ações dos profissionais de saúde na visita domiciliar ao binômio mãe-bebê. Rev Enferm UERJ. 2016; 24(5):1-6.
• ⁸
  Andrade RD, Santos JS, Maia MA, Mello DF. Factors related to women’s health in puerperium and repercussions on child health. Esc Anna Nery. 2015; 19(1):181-6.
• ⁹
  Brasil. Ministério da Saúde. Manual AIDPI Neonatal Série A. Normas e Manuais Técnicos. Brasília (DF): Ministério da Saúde; 2014.
• ¹⁰
  Fujimori E, Higuchi CH, Cursino EG, Veríssimo MLÓR, Borges ALV, Mello DF, et al. Ensino da estratégia Atenção Integrada às Doenças Prevalentes na Infância na graduação em enfermagem. Rev Lat Am Enfermagem. 2013; 21(3):1-8.
• ¹¹
  Carvalho LM, Anjos DS, Rozendo CA, Costa LM. Agenda de compromisso para a saúde integral e redução da mortalidade infantil em um município de alagoas. Rev Bras Promoc Saude. 2013; 26(4):530-8.
• ¹²
  Brasil. Ministério da Saúde. Secretaria de Atenção à Saúde. Departamento de Ações Programáticas Estratégicas. Agenda de compromissos para a saúde integral da criança e redução de mortalidade infantil. Série, Normas e Manuais Técnicos. Brasília (DF); Ministério da Saúde; 2004.
• ¹³
  Hasson F, Keeney S. Enhancing rigour in the Delphi technique research. Technol Forecast Soc Change. 2011; 78:1695-704.
• ¹⁴
  Brasil. Ministério da Saúde. Secretaria de Atenção à Saúde. Departamento de Ações Programáticas Estratégicas. Atenção à saúde do recém-nascido: guia para os profissionais de saúde. 2a ed. Brasília (DF); Ministério da Saúde; 2012.
• ¹⁵
  Moreira MD, Gaiva MA. Abordagem do contexto de vida da criança na consulta de enfermagem. Fundam Care Online. 2017; 9(2):e432-440.
• ¹⁶
  Almeida MA, Pergher AK, Canto DF. Validation of mapping of care actions prescribed for orthopedic patients onto the nursing interventions classification. Rev Lat Am Enfermagem. 2010; 18(1):1-8.
• ¹⁷
  Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951; 16(3):297-334.
• ¹⁸
  Requejo JH, Bhutta ZA. The post-2015 agenda: staying the course in maternal and child survival. Arch Dis Child. 2015; 100(Suppl1):76–81.
• ¹⁹
  Reichert AP, Rodrigues PF, Cruz TM, Dias TK, Tacla MT, Collet N. Percepção de mães sobre o vínculo com enfermeiros na consulta à criança. Rev Enferm UFPE Online. 2017; 11(2):e483-90.
• ²⁰
  Brasil. Ministério da Saúde. Manual de vigilância do óbito infantil e fetal e do Comitê de Prevenção do Óbito Infantil e Fetal. 2a ed. Brasília (DF); Ministério da Saúde; 2009.
• ²¹
  Leal GC, Wolff, LD, Gonçalves LS, Peres A M, Oliveira FA. Práticas de atenção perinatal em maternidades de risco habitual: avaliação na perspectiva de mulheres. Cogitare Enferm. 2016; 21(2):1-8.

Publication Dates

• Publication in this collection
  06 July 2018
• Date of issue
  Mar-Apr 2018

History

• Received
  14 Oct 2017
• Accepted
  6 Mar 2018