Instrument for evaluating child hearing health services: construction and validity ^*

ABSTRACT

Objective

To validate a matrix of indicators to assess the structure and process of child hearing health services.

Method

A study of semantic and content validation with specialists in the area of Nursing and Speech therapy. The instrument contained 20 indicators with scores to be analyzed by experts. The condition for the validity suitability of each indicator and scores were an Item-level Content Validity Index (I-CVI) of 0.80 and a Scale-level CVI (S-CVI) of 0.80.

Results

Twenty-two (22) specialists participated in the study, with 59% being nurses and 41% speech therapists, of which 32% had specializations, 45% had a Master’s degree, 18% had a Doctorate degree and 5% had a Post-doctorate degree. The mean I-CVI and S-CVI of the indicators evaluated as suitable were 0.96, while for the suggested scores the I-CVI was 0.80 and the S-CVI was 0.82.

Conclusion

The matrix of indicators was considered valid for evaluating child hearing health services.

Child Health; Hearing Loss; Primary Care Nursing; Health Evaluation; Validation Studies

RESUMO

Objetivo

Validar uma matriz de indicadores para avaliação de estrutura e processo em serviços de saúde auditiva infantil.

Método

Estudo de validação semântica de conteúdo com especialistas da área de Enfermagem e Fonoaudiologia. O instrumento continha 20 indicadores com pontuações a serem analisadas pelos especialistas. A condição para a adequação da validade de cada indicador e pontuação foi o Índice de Validação de Conteúdo por Item (I-IVC) de 0,80 e o IVC por nível de Escala (S-IVC) de 0,80.

Resultados

Participaram da pesquisa 22 especialistas. 59% eram enfermeiros e 41% fonoaudiólogos, sendo 32% especialistas, 45% mestres, 18% doutores e 5% pós-doutores. As médias dos I-IVC e S-IVC dos indicadores avaliados como adequados foi de 0,96, e nas pontuações sugeridas o I-IVC foi de 0,80 e o S-IVC de 0,82.

Conclusão

A matriz de indicadores foi considerada válida para avaliação de serviço de saúde auditiva infantil.

Saúde da Criança; Perda Auditiva; Enfermagem de Atenção Primária; Avaliação em Saúde; Estudos de Validação

RESUMEN

Objetivo

Validar una matriz de indicadores para evaluación de estructura y proceso en servicios de salud auditiva infantil.

Método

Estudio de validación semántica de contenido con expertos del área de Enfermería y Fonoaudiología. El instrumento contenía 20 indicadores con puntajes que serían analizadas por los expertos. La condición para la adecuación de la validez de cada indicador y puntaje fue el Índice de Validación de Contenido por Ítem (I-IVC) de 0,80 y el IVC por nivel de Escala (S-IVC) de 0,80.

Resultados

Participaron en la investigación 22 expertos. El 59% eran enfermeros y el 41% fonoaudiólogos, siendo el 32% con especialización, el 45% con máster, el 18% doctores y el 5% post doctores. Los promedios de los I-IVC y S-IVC de los indicadores evaluados como adecuados fueron de 0,96 y en los puntajes sugeridos el I-IVC fue de 0,80 y el S-IVC de 0,82.

Conclusión

La matriz de indicadores fue considerada válida para evaluación de servicio de salud auditiva infantil.

Salud del Niño; Pérdida Auditiva; Enfermería de Atención Primaria; Evaluación en Salud; Estudios de Validación

INTRODUCTION

In Brazil, 5% of the general population has hearing loss, which corresponds to approximately 9.7 million people⁽¹⁾. Hearing losses may be associated with several risk factors in neonates and infants such as family history, congenital infections, postnatal infections, craniofacial anomalies, head trauma, hyperbilirubinemia with exchange transfusion, ototoxic medications, low birth weight, prematurity, length of stay in Neonatal Intensive Care Units (NICU), mechanical ventilation, suspected family developmental, hearing, speech or language delay, among others^(2-4).

The National Care Policy for Hearing Health ( Política Nacional de Atenção à Saúde Auditiva ) aims at successful interventions through actions of health promotion, prevention, treatment and rehabilitation of diseases/injuries, determining the technical quality required for good performance at all levels of health care through the interdisciplinary team and intersectoral actions^(5-6).

In the scope of Primary Care and as participants in the Family Health Strategy team (FHS), nurses perform the systematic follow-up of children (0 to 5 years old) during routine consultations by assessing their growth and development, vaccination, breastfeeding, complementary feeding and providing guidance to mothers or caregivers on the prevention of accidents, individual/environmental hygiene and early identification of diseases for appropriate intervention⁽⁷⁾.

Child development milestones are evaluated during childcare consultations, and behavioral measures are adopted in case of possible changes, which should be based on the diagnostic impression of probable developmental delay, development alert, normal development with risk factors or normal development, as described in the monitoring instrument for development of the Child Health Handbook^(8-12).

Childcare follow-ups allow nurses to identify the absence of milestones related to children’s hearing health early on. Proper practice by this professional in childcare may reduce the effects of hearing loss. Normative evaluation in the health service enables investigating how primary care nurses act in their attention to children’s hearing health, offering subsidies for planning and managing health services⁽¹³⁾.

A normative evaluation consists in judging an intervention based on criteria and norms, comparing the employed resources and the organization (structure), the services produced (process) and the obtained results⁽¹⁴⁾.

In order to evaluate the structure and process of a child hearing health service, encompassing nursing practice, it is necessary to validate the proposed instrument in order to increase the reliability of the results, since the content validity evidences the specialized judges’ measurement of the instrument’s representativeness for the study population and its correspondence with the proposed objectives in the research^(15-16). Thus, the objective of this study was to validate a matrix of indicators to evaluate the structure and process in child hearing health services.

METHOD

A methodological study carried out for semantic and content validation of a matrix of indicators to evaluate the structure and process in child hearing health services.

Validity is composed of a parameter that is congruent with the measured characteristic of the objects⁽¹⁷⁾. The semantic validation verifies the experts’ understanding of the instrument items and the possible needs for modifications, seeking to increase understandability⁽¹⁶⁾. Content validation is the proportion of specialists who rank each item according to its relevance or suitability⁽¹⁵⁾.

The sample was intentional, consisting of 22 specialists graduated in Nursing or Speech therapy, with at least one post-graduation (specialization, Master’s, Doctorate and/or Post-doctorate degree), working in the area of childcare, primary care or neonatal hearing screening. The sample size calculation was based on the formula that considers the final proportion of specialists related to a dichotomous variable with the maximum acceptable difference of this proportion. A minimum proportion of 85% agreement between the experts and a 15% difference in this agreement was adopted. The formula used was: n =Zα ² . P.(1- P) / d² , considering a 95% confidence level, resulting in 22 experts⁽¹⁸⁾.

An invitation letter was sent to the e-mail of 26 experts, explaining the purpose of validating the matrix of indicators, and 22 of these confirmed their participation (13 nurses and 9 speech therapists). The face-to-face interview was performed upon prior scheduling with each specialist to sign the clear and Informed Consent Form (ICF) and an analysis of the evaluation form, which occurred between July and August 2013. After consolidating the corrections/suggestions resulting from the first analysis of experts in the matrix of indicators, the corrected instrument was sent to the e-mails of each specialist for a second evaluation, which resulted in no further suggestions regarding the content and semantics of aforementioned instrument.

The matrix of indicators originated from a logical model developed by the researchers based on the following references: National Care Policy on Child Hearing Health ( Política Nacional de A tenção à Saúde Audit iva Infantil ), report of the Joint Committee on Infant Hearing (JCIH), National Policy for Basic Care ( Polític a Nacional de Atençã o Básica ), Handbook of Integrated Care for Prevalent Childhood Illnesses ( Manual de A tenção Integrada às Doenças Prevalentes na Infância – AIDPI) and Child Health Handbook^{(3,5,9-10,19)}.

The matrix contemplates three dimensions: 1) Physical structure and necessary materials; 2) Technical and scientific suitability; and 3) Professional training. The established criteria correspond to attributes to measure the components of a service interconnected to indicators, which reach an expected score when met⁽¹⁴⁾. First, 20 indicators were created by the researchers, 9 for the physical structure dimension and 11 for the other two dimensions. The scores initially suggested by the researchers were distributed as 40 points for physical structure and necessary materials (4.4 points per criterion), and 60 points for technical-scientific suitability and professional training (3.1 points per criterion), totaling 100 points.

For the technical-scientific suitability dimension in the sub-dimension of monitoring hearing development milestones and conduct, one of the indicators corresponds to four clinical cases, and 5.0 points were suggested for the correct conduct in each clinical case, totaling 20.0 points. Clinical cases were constructed by the researchers according to the diagnostic impression of probable developmental delay, development alert, normal development with risk factors, or normal development, according to the instrument for monitoring development of the Child Health Handbook⁽⁸⁾. The four suggested clinical cases are described below:

Case 1 – A 2-month-old child has a family history of deafness and did not perform the Neonatal Hearing Screening. On the current evaluation, the child does not react to the sound stimulus performed about 30 cm from the ear. The conduct will be forwarding him/her for evaluation with other professionals and providing guidance to the caregiver.

Case 2 – A 6-month-old child presenting a previous history of meningitis at 5 months performed the Neonatal Hearing Screening in the first month of life, with no alterations in the results. On the current evaluation, the child reacts to the soft sound stimulus by turning its head toward the sound. The conduct will be providing guidance to the caregiver on the warning signs and reassessing the child in 30 days.

Case 3 – An 8-month-old child has no risk factors for hearing health, as he/she performed the Neonatal Hearing Screening without any changes in the result. On the current evaluation, the child reacts to the conversation by saying “ da da ... da da ... ”. The conduct will be to guide the caregiver to (Continue) stimulating the child and to schedule the next routine appointment.

Case 4 – A 1-month-old child with a birth weight of 2,300 g remained hospitalized for 15 days in the NICU shortly after birth due to respiratory failure and seizure episodes. The Neonatal Hearing Screening was performed after hospital discharge, presenting alterations in its result. In the present evaluation, the child does not show any response to the sound stimulus about 30 cm from the ear. The conduct will be directing the child to a neuropsychomotor evaluation and to provide guidance to the caregiver.

Each item in the indicator matrix was evaluated for the appropriate nomenclature, clarity, objectivity and applicability according to the Likert scale, adapted as: unsuitable, not very suitable, suitable and very suitable. The conditions for validating each indicator and the suggested scores were given when the mean of the Item-level Content Validity Index (I-CVI) was greater than or equal to 0.80, and the Scale-level Content Validity Index (S-CVI) was 0.80, in order to establish excellence in content validity⁽²⁰⁾. Modifications were made to content, the text and the grammar when the indicator was considered unsuitable by the experts, and the scores were redistributed between the criteria.

The study was approved by the Ethics and Research Committee of the Health Sciences Center of UFPE, opinion number 511.566 (01/15/2014), in compliance with the norms established by Resolution 466/2012 of the National Health Council – Ministry of Health⁽²¹⁾.

RESULTS

Of the 22 specialists, 59% were nurses and 41% were speech therapists, 32% had specific training in child health, 45% had Master’s degrees, 18% had a PhD and 5% had a post-doctorate degree. The indicators classified as partially or completely suitable obtained a mean for both I-CVI and S-CVI equal to 0.96, and in the scores classified as fairly or completely suitable, the mean of the I-CVI was 0.80, and for the S-CVI it was 0.82 ( Table 1 ).

Table 1
– Indicators and suggested scores classified as partially or completely suitable – Recife, PE, 2013. ((Continue)s)

Aspect: Physical structure
Dimension: Physical structure and necessary materials
Considered criteria	Indicators	I-CVI (Indicators)	Suggested scores	I-CVI (Scores)
Physical structure
Room for consultations	At least one room for the Family Health Team to carry out childcare consultations	1.00	4.4	0.86
Space for educational activities	At least one space destined for educational activities in the Health Unit	0.95	4.4	0.81
Necessary materials
Tables	Availability of a table to assist in registering childcare consultations and educational activities	0.95	4.4	0.86
Chairs	Availability of chairs for the accommodation of professionals, patients and companions.	1.00	4.4	0.90
Beds or stretchers	Availability of beds or stretchers for evaluating development milestones	0.90	4.4	0.86
Instruments and/or strategies used to emit sounds	Availability of percussion instruments (bells or rattles) and/or strategies used to evaluate hearing and language milestones	0.86	4.4	0.72
Child Health Handbook	Availability of Child Health Handbook available at the service for applying the development monitoring instrument	0.95	4.4	0.81
Registration form/child’s medical record/Child Health Handbook with the milestones	Availability of registration form/medical record/Child Health Handbook with evaluation of hearing and language milestones	0.86	4.4	0.77
Informative materials	Availability of informative materials for educational activities on children’s hearing health in the community and health staff (folders, booklets, brochures, etc.)	1.00	4.4	0.81
Aspect: Process
Dimension: Technical-scientific suitability
Considered criteria Indicators	Indicators	I-CVI (Indicators)	Suggested scores	I-CVI (Scores)
Sub-dimension: Management of risk factors for hearing loss		-	-	-
Identification of risk factors	Number of risk factors for hearing loss identified by the nurse	1.00	3.1	0.86
Association of risk factors with the development milestones	Checking the association of risk factors for hearing loss with child development milestones in professional practice	0.95	3.1	0.86
Sub-dimension: Monitoring the hearing development milestones and conduct		-	-	-
Direct hearing development milestones	Identifying at least three of the development milestones related to hearing in the described items	0.95	3.1	0.77
Indirect hearing development milestones	Identifying at least three of the development milestones indirectly related to hearing in the described items	0.95	3.1	0.77
Beginning of Hearing Assessment	Identifying the most likely age for assessment (NHS) and diagnosis of hearing loss with a qualified professional Identifying the most favorable age for hearing loss intervention/rehabilitation with a qualified professional	0.86	3.1 x 5	0.72
Newborn Hearing Screening (NHS)
Trained professional to perform the NHS
Diagnosis of hearing loss
Hearing loss intervention/rehabilitation
Sub-dimension: Monitoring hearing development milestones and conduct		-	-
Case 1	Correctly evaluating at least two cases related to the possibility of hearing loss, the child’s hearing development milestones and the adopted conduct	1.00	5.0	0.90
Case 2		1.00	5.0	0.90
Case 3		1.00	5.0	0.90
Case 4		1.00	5.0	0.90
Sub-dimension: Educational practices		-	-
Carrying out educational activities on child hearing health for the community	Carrying out at least one educational activity for the community	1.00	3.1	0.72
Continuous education on child hearing health for the health team	Carrying out at least one continuous educational activity for the Family Health Team	1.00	3.1	0.68
Dimension: Professional training
Participating in courses related to child hearing health	Participating in at least one course with content related to child hearing health	1.00	3.1	0.72
Participating in courses and/or training in child hearing healthcare	Participating in at least one course and/or training focused on child hearing healthcare	0.95	3.1	0.68
-	Total I – CVI 0.96	Total I – CVI		0.80
-	Total S – CVI 0.96	Total S – CVI		0.82

The S-CVI scores were considered valid based on the recommendations of the literature as this index was greater than 0.80⁽²⁰⁾. According to the experts’ evaluation, an indicator of the technical-scientific suitability dimension in the sub-dimension of educational practices was excluded (Continuous education on child hearing health for the health team – Carrying out at least one continuous educational activity for the Family Health Team), and two more indicators were added in the sub-dimension monitoring the development of hearing and conduct development milestones (Registering in the chart/medical record/Child Health Handbook on the hearing milestones – Identifying at least one chart, medical record or Child Health Handbook with the records of hearing milestones; Using percussion instruments (rattles, bells, etc.) and/or other strategies (clapping hands, snapping fingers, etc.) to emit sounds – Verifying the use of percussion instruments (rattles, bells, etc.) and/or strategies (clapping hands, snapping fingers, etc.) to evaluate hearing development milestones), which resulted in 21 indicators. Chart 1 shows the matrix with the validated indicators and scores to be applied in evaluating the structure and process in child hearing health services, also taking into account the practice of Family Health Strategy nurses.

Chart 1
– Matrix of validated indicators for evaluating the structure and process in child hearing health services, considering nurses’ practice - Recife, PE, 2013.

	Dimensions	Sub-dimensions	Considered criteria	Indicators	Verification Parameters (Expected scores)	Verification Parameters (Cut-off points)
Structure	Physical structure and necessary materials	-	Physical structure	-	-	-
			Room for consultations	At least one room for the Family Health Team to carry out childcare consultations	4.4	4.4 points = if the room for consultation is available 0 points = if the room for consultation is not available
			Space for educational activities	At least one space destined for educational activities in the Health Unit	4.4	4.4 points = if the space for educational activities is available 0 points = if the space for educational activities is not available
			Necessary materials	-	-	-
Structure	Physical structure and necessary materials	-	Table	Availability of a table to assist in registering childcare consultations and educational activities	4.4	4.4 points = if a table is available 0 points = if a table is not available
			Chairs	Availability of chairs for accommodating professionals, patients and companions.	4.4	4.4 points = if two or more chairs are available 0 points = if no chairs are available
			Bed, stretcher or exam table	Availability of a bed, stretcher or exam table for evaluating the development milestones	4.4	4.4 points = if a bed or stretcher is available 0 points = if no beds or stretchers are available
			Instruments and/or strategies used to emit sounds	Availability of percussion instruments (bells or rattles) used to evaluate hearing and language development milestones	4.4	4.4 points = if percussion instruments are available 0 points = if percussion instruments are not available
			Child Health Handbook	Availability of the Handbook at the service for applying the monitoring instrument for development	4.4	4.4 points = if the Handbooks are available; 0 points = if the Handbooks are not available
			Registration form / child’s medical record / Child Health Handbook with the development milestones	Availability of the registration form/medical record/Child Health Handbook with assessment of hearing and language development milestones	4.4	4.4 points = if the registration form/Handbook/child’s medical record is available 0 points = if the registration form/ Handbook/child’s medical record is not available
			Informative materials	Availability of informative materials for educational activities on child development, including those on hearing and language, aimed at the community and the health staff (folders, booklets, brochures, etc.)	4.4	4.4 points = if informative materials are available 0 points = if informative materials are not available
Process	Technical-scientific suitability	Management of risk indicators for hearing loss	Identification of risk indicators	The nurse identifies at least two risk indicators for hearing loss reported in the Child Health Handbook	3.1	3.1 points = if at least two risk indicators have been described 0 points = if at least two risk indicators have not been described
			Association of risk indicators with the development milestones	Checking for hearing loss associated with child development milestones in professional practice	3.1	3.1 points = if the answer is “yes” 0 points = if the answer is “no”
		Monitoring the development of hearing milestones and conduct	Registering in the chart/medical record/Child Health Handbook the hearing development milestones	Identifying at least one chart, medical record or Child Health Handbook with the records of hearing development milestones	3.1	3.1 points = if any of the criteria is described 0 points = if none of the criteria is described
			Using instruments and/or strategies for emitting sound	Verifying the use of percussion instruments (rattles, bells, etc.) and/or strategies (clapping, snapping fingers, etc.) to evaluate hearing development milestones	3.1	3.1 points = if one of the criteria is described 0 points = if none of the criteria is described
			Direct hearing development milestones	Identifying at least three of the development milestones related to hearing in the described items	3.1	3.1 points = if all three criteria items are correct 0 points = if the three criteria items are not correct
			Reaction to sound when stimulated
			Social smile when stimulated
			Active response to social contact
			Locates sound when stimulated
			Recognizes two actions by pointing to the images when interrogated
			Indirect hearing development milestones	Identifying at least four of the development milestones indirectly related to hearing in the described items	3.1	3.1 points = if all four criteria items are correct 0 points = if the four criteria items are not correct
			Emission of sounds
			Duplication of syllables
			Production of “jargon” and/or incomprehensible conversation
			Vocalizes one word
			Vocalizes three words
			Understands sentences with two words
Process	Technical-scientific suitability	Monitoring the development of hearing and conduct milestones and conduct	Beginning of Hearing Assessment	Identifying the most likely moment for assessment (NHS) of hearing loss with a qualified professional	3.1	3.1 points = if the criterion was described 0 points = if no criterion was described
			Newborn Hearing Screening (NHS)		3.1	3.1 points = if the criterion was described 0 points = if no criterion was described
			Diagnosis of hearing loss	Identifying the most favorable moment for hearing loss intervention/rehabilitation with a qualified professional	3.1	3.1 points = if the criterion was described 0 points = if no criterion was described
			Hearing loss intervention/rehabilitation		3.1	3.1 points = if the criterion was described 0 points = if no criterion was described
			Resolution of case 1	Correctly evaluating at least two cases related to the possibility of hearing loss and child’s hearing development milestones	20	20 points = if all the affirmations of the cases are correct 5.0 points = if some of the affirmations of the cases are correct 0 points = if none of the affirmations of the cases are correct
			Resolution of case 2
			Resolution of case 3
			Resolution of case 4
		Educational practices	Carrying out educational activities in child hearing health for the community	Carrying out at least one educational activity on child hearing health for the community in the previous year	3.1	3.1 points = if the answer is “yes” 0 points = if the answer is “no”
	Professional training	-	Carrying out courses with content related to child hearing health	Participating in at least one course with content related to child hearing health	3.1	3.1 points = if the answer is “yes” 0 points = if the answer is “no”
			Participating in courses and/or training in child hearing health care	Participating in at least one course and/or training focused on child hearing health care	3.1	3.1 points = if the answer is “yes” 0 points = if the answer is “no”

DISCUSSION

Six of the nine indicators for the physical structure/necessary materials dimension reached an I-CVI > 90%. Of the three indicators with an I-CVI < 90%, the term examination table was included in the indicator of beds or stretchers, and the term Child Health Handbook was removed from the registration form/child’s medical record/Child Health Handbook, since it is described as one of the criteria related to the nine indicators. This last indicator raised questions regarding the importance of recording the information on printed forms requested during the child care consultation, thereby leading to introduction of a new indicator in the technical-scientific suitability dimension, sub-dimension of monitoring the hearing development milestones and conduct to evaluate the hearing development milestones recorded in the forms/printouts (registration form, medical record or Child Health Handbook) ( Chart 1 ).

The indicator percussion instruments (bells or rattles) and/or strategies used to evaluate hearing and language milestones had a disagreement of 14% among the experts on the term “strategies”, due to the fact that this term cannot be included in the physical structure, but rather in the evaluation process; therefore it was removed from the physical structure and inserted into the process as a new indicator entitled “Using percussion instruments (rattles, bells, etc.) and/or (other) strategies (clapping hands, snapping fingers, etc.) to emit sounds in the sub-dimension of monitoring hearing development milestones and conduct (Chart 1). These modifications had the purpose of increasing the comprehensibility of the instrument for evaluating the structure and the process in child hearing health services, considering nurses’ practice in childcare. The construction of indicators interconnected to well-established criteria in a normative evaluation aims to offer reliability to the evaluation process, thus enabling the results of the analyzed aspects to show the reality of the evaluated service/program⁽¹⁴⁾. In the Family Health Strategy context, a normative evaluation of the physical structure and material resources of prenatal care carried out in seven Family Health Units (FHU) in the city of Rio de Janeiro demonstrated the consonance of the indicators and adopted criteria to support professional practice in its totality⁽²²⁾.

With regard to the physical structure/necessary materials dimension, only the indicator related to the availability of chairs for accommodating professionals, patients and companions reached 90% agreement among the experts; while an 86% agreement was found for the indicators regarding the room/space for consultations and beds or stretchers, followed by 81% for the availability of space suitable for educational activities, Child Health Handbook and Informative materials.

A greater disagreement between the experts was observed for the indicator instruments used to emit sounds and for the registration form, medical records and the Child Health Handbook of 28% and 23%, respectively. The disagreement in relation to the instruments used to emit sounds may be due to the fact that the experts consider this criterion specific for child hearing health, and the possibility that these instruments are not available among the basic materials used in the FHU, as shown by a normative evaluation in Brazilian states that did not adopt the existence of this indicator in its criteria among the basic materials for performing the activities in the FHS⁽²³⁾.

Eight of the 11 indicators in the dimension of technical-scientific suitability and professional training reached an I-CVI ≥ 95% agreement among the experts. Only one indicator, which addresses the identification of the most favorable age for the initial evaluation (NHS), hearing loss diagnosis and intervention/rehabilitation, of the sub-dimension for monitoring the hearing development milestones reached an I-CVI of 86%. This indicator presented disagreement (among the experts) regarding the most favorable/probable age due to the term “age” corresponding to an exact measurement which may lead the nurse to a misunderstanding in the evaluation, with recommendations to replace the term by “most favorable moment”.

The four clinical cases evaluated separately obtained 100% agreement among the experts. It is worth pointing out that some changes in grammar have been made to make them clearer, more objective, applicable and appropriate. The four clinical cases validated according to the diagnostic impressions of the child development milestones evaluation can be applied in other settings to evaluate the practice of primary care nurses with the aim of highlighting the difficulties and to support possible improvements within the scope of child hearing health care. Application of the clinical cases in other studies will enable an understanding of the effectiveness of nurses’ practice in evaluating child development milestones related to hearing health in order to make it a consolidated practice in nurses’ performance.

Validating indicators and clinical cases for evaluating the structure and the process including nurses’ practice in child hearing health services is important, as the FHS acts as the entry point of the health system in Primary Care through actions of health promotion and prevention of diseases/injuries during the childhood period, favoring implementation of the National Policy on Hearing Health Care ( Política Nacional d e Atenção à Saúde Au ditiva ) to improve the diagnostic and therapeutic assistance of neonates and infants with hearing loss^(5,24-25).

In the sub-dimension educational practices, the indicator continuous education in child hearing health for the health team was classified as not very suitable or unsuitable by 32% of the experts, as the matrix of indicators was directed towards nurses rather than the health team. However, this indicator was disregarded, as nurses continuous education is contemplated by the indicators described in the professional training sub-dimension.

The continuous education indicator demonstrates nurses’ knowledge about child hearing health acquired during their initial graduation, post-graduation and training, and it is directly associated with the performance of educational practices during their professional actuation. Not only nurses but all FHS professionals who act as health educators should be responsible for the systematization, development and monitoring of the educational process that values human beings in its biopsychosocial aspects, thereby seeking better results according to the reality of the population served at the FHU^(26-27).

Regarding the scores of the technical-scientific suitability dimension, a 90% agreement was achieved regarding the suggested clinical cases, reinforcing the importance of a higher score for this indicator in relation to the others in this dimension. The conduct of clinical cases is a relevant indicator for evaluating the structure and the process in child hearing health services, within the framework of the Family Health Strategy nurses’ practice.

In relation to the indicator matrix scores, in the sub-dimension management of risk factors for hearing loss, the indicators for identification of risk factors for hearing loss and the association of these factors with child development milestones reached an agreement of 86%, emphasizing the need for nurses to be aware of some of the risk factors for hearing loss described by the Joint ⁽³⁾, since they can be identified in monitoring hearing and language development in childcare consultations.

In relation to developmental milestones, 23% of the experts disagreed on the score, arguing that it should be higher than the other evaluated indicators. Also, 28% of the experts disagreed for the initial evaluation criteria (NHS), trained professionals, diagnosis, hearing loss intervention/rehabilitation represented in two indicators (excluding the trained professional for the NHS) for not considering the necessity of the suggested scoring in this criterion, and in the indicator score for the performance of educational practices in child hearing health in the community they can be very specific for evaluating structure and process, considering the practice of the FHS nurses. It is known that performing health education is part of the attributions of primary health care nurses, which can be directed individually or collectively to develop the responsibility of family members on child health, and consequently to promote improvement in prognostics of child hearing health⁽²⁸⁾.

In the professional training dimension, disagreement by the experts on the scores was 28% for participation in courses related to the child hearing health, and 32% for specific courses and training. These percentages may be due to the experts questioning the Nursing training offered by various educational institutions and the availability of courses/training offered by the health secretariats, which may be incipient regarding child hearing health.

The suggested score could be lower in relation to the reality experienced by FHS nurses, considering that the educational practices are based on knowledge acquired during professional training, and in the routines and demands of the FHU to meet the objectives of the National Primary Health Care Policy^(25-26). One of the roles of health management is to offer courses and training for professionals in order to improve the care quality provided to children’s health, especially with regard to child hearing health, in order to reinforce the recommendations of the National Policy on Hearing Health Care⁽⁵⁾.

Another aspect to be considered was the indicator of nurses’ participation in continuous education for the health team, with 32% disagreement among experts due to the questions raised in the indicators of the professional training dimension. Therefore, this indicator has been removed from the technical-scientific suitability dimension since it covers the continuous education of the health team, and the purpose of the instrument is to evaluate the structure and the process in child hearing health services within the framework of nursing practice.

CONCLUSION

Validation of the matrix of indicators for evaluating the structure and the process, including nurses’ practice in child hearing health services was fundamental to increase the understanding of this professional category during data collection and to facilitate the evaluation process. The experts’ opinions and suggestions were analyzed according to the I-CVI values, and most of the indicators obtained an I-CVI agreement ≥ 80%. A higher disagreement among the experts was observed in relation to the suggested scores, since the values for each indicator were analyzed in relation to the evaluated aspect. The scores remained uniformly distributed among each aspect to be evaluated, and in agreement with the S-CVI values suggested by the literature.

The matrix to be analyzed initially consisted of 20 indicators and 21 indicators were validated at the end, in addition to the four clinical cases. Only two indicators were added and one was disregarded after the experts’ analysis. The four clinical cases can be applied with other nurses to evaluate the effectiveness of this professional category’s practice in evaluating child development milestones related to hearing health. Thus, the matrix of indicators was considered valid for evaluating child hearing health services. Furthermore, the validation of this matrix of indicators for evaluating the structure and the process, taking into consideration nurses’ practice in child hearing health services will enable this tool to be applied in other studies.

REFERÊNCIAS

• ¹
  Instituto Brasileiro de Geografia e Estatística. Censo demográfico 2010 [Internet]. Rio de Janeiro: IBGE; 2012 [citado 2014 set. 02]. Disponível em: http://www.ibge.gov.br/home/estatistica/populacao/censo2010/resultados_gerais_amostra/default_resultados_gerais_amostra.shtm
  » http://www.ibge.gov.br/home/estatistica/populacao/censo2010/resultados_gerais_amostra/default_resultados_gerais_amostra.shtm
• ²
  Azevedo MF. Triagem auditiva neonatal. In: Ferreira LP, Befi-Lopes DM, Limongi SCO, organizadores. Tratado de fonoaudiologia. São Paulo: Roca; 2004. p. 604-16.
• ³
  American Academy of Pediatrics, Joint Committee on Infant Hearing. Year 2007. Position statement: principles and guidelines for early hearing detection and intervention programs. Pediatrics. 2007;120(4):898-921.
• ⁴
  Lopes MKD, Santos TMM. Comparison of indicators of risk of deafness in newborns studied in the years 1995 and 2005. Arq Int Otorrinol [Internet]. 2011 [cited 2015 Mar 24];15(1):35-40. Available from: http://www.arquivosdeorl.org.br/conteudo/acervo_port.asp?id=738
  » http://www.arquivosdeorl.org.br/conteudo/acervo_port.asp?id=738
• ⁵
  Brasil. Ministério da Saúde. Portaria n. 2073, de 28 de setembro de 2004. Política Nacional de Atenção à Saúde Auditiva [Internet]. Brasília; 2004 [citado 2014 maio 12]. Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2004/prt2073_28_09_2004.html
  » http://bvsms.saude.gov.br/bvs/saudelegis/gm/2004/prt2073_28_09_2004.html
• ⁶
  Brasil. Ministério da Saúde. Portaria n. 587, de 07 de outubro de 2004. Política Nacional de Atenção à Saúde Auditiva [Internet]. Brasília; 2004 [citado 2014 abr. 12]. Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/sas/2004/prt0587_07_10_2004.html
  » http://bvsms.saude.gov.br/bvs/saudelegis/sas/2004/prt0587_07_10_2004.html
• ⁷
  Campos RMC, Ribeiro CA, Silva CV, Saparolli ECL. Nursing consultation in child care: the experience of nurses in the Family Health Strategy. Rev Esc Enferm USP [Internet]. 2011 [cited 2014 Sep 01];45(3):565-73. Available from: http://www.scielo.br/pdf/reeusp/v45n3/en_v45n3a03.pdf
  » http://www.scielo.br/pdf/reeusp/v45n3/en_v45n3a03.pdf
• ⁸
  Brasil. Ministério da Saúde. Caderneta de Saúde da Criança – Passaporte da Cidadania. 7ª ed. [Internet]. Brasília; 2011 [citado 2014 set. 17]. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/caderneta_saude_crianca_menina_7ed.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/caderneta_saude_crianca_menina_7ed.pdf
• ⁹
  Brasil. Ministério da Saúde. Saúde da criança: crescimento e desenvolvimento [Internet]. Brasília; 2012 [citado 2014 set. 17]. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/crescimento_desenvolvimento.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/crescimento_desenvolvimento.pdf
• ¹⁰
  Brasil. Ministério da Saúde, Organização Pan-americana de Saúde. Quadro de procedimentos [Internet]. Brasília; 2012 [citado 2014 out. 16]. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/maual_aidpi_neonatal_quadro_procedimentos.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/maual_aidpi_neonatal_quadro_procedimentos.pdf
• ¹¹
  Brasil. Ministério da Saúde. AIDPI - Atenção Integrada às Doenças Prevalentes na Infância [Internet]. Brasília; 2003 [citado 2014 dez. 21]. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/AIDPI_modulo_1.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/AIDPI_modulo_1.pdf
• ¹²
  Santos MEA, Quintão NT, Almeida RX. Evaluation of the mark of child development according to strategy integrated management of childhood illness. Esc Anna Nery [Internet]. 2010 [cited 2015 Jul. 20]; 14 (3): 591-598. Available from: http://www.scielo.br/pdf/ean/v14n3/v14n3a22.pdf DOI: http://dx.doi.org/10.1590/S1414-81452010000300022
  » http://www.scielo.br/pdf/ean/v14n3/v14n3a22.pdf» http://dx.doi.org/10.1590/S1414-81452010000300022
• ¹³
  Sancho LG, Dain S. Avaliação em saúde e avaliação econômica em saúde: introdução ao debate sobre seus pontos de interseção. Ciênc Saúde Coletiva [Internet]. 2012 [citado 2014 dez. 21];17(3):765-74. Disponível em: http://www.scielo.br/pdf/csc/v17n3/v17n3a24.pdf
  » http://www.scielo.br/pdf/csc/v17n3/v17n3a24.pdf
• ¹⁴
  Samico I, Felisberto E, Figueiró AC, Frias PG. Avaliação em saúde: bases conceituais e operacionais. Rio de Janeiro: MedBook; 2010.
• ¹⁵
  Polit DF, Beck, CT. Fundamentos da pesquisa em enfermagem: avaliação das evidências para a prática de enfermagem. 7ª ed. Porto Alegre: Artmed; 2011.
• ¹⁶
  Galdeano LE, Furuya RK, Delacio MCB, Dantas RAS, Rossi LA. Validação semântica do Cardiac Patients Learning Needs Invento ry para brasileiros e portugueses. Rev Gaúcha Enferm [Internet]. 2011 [citado 2015 maio 14];32(3):602-10. Disponível em: http://www.scielo.br/pdf/rgenf/v32n3/24.pdf
  » http://www.scielo.br/pdf/rgenf/v32n3/24.pdf
• ¹⁷
  Pasquali L. Psicometria: teoria dos testes na psicologia e na educação. 4ª ed. Rio de Janeiro: Vozes; 2011.
• ¹⁸
  Lopes MVO, Silva VM, Araujo TL. Methods for establishing the accuracy of clinical indicators in predicting nursing diagnoses. Int J Nurs Knowl. 2012;23(3):134-9.
• ¹⁹
  Brasil. Ministério da Saúde. Portaria n. 2.488, de 21 de outubro de 2011. Aprova a Política Nacional de Atenção Básica, estabelecendo a revisão de diretrizes e normas para a organização da Atenção Básica, para a Estratégia Saúde da Família (ESF) e o Programa de Agentes Comunitários de Saúde (PACS) [Internet]. Brasília; 2011 [citado 2015 nov. 17]. Disponível em: http://www.saude.al.gov.br/sites/default/files/nova_pnab_-_portaria_no_2488_0.pdf
  » http://www.saude.al.gov.br/sites/default/files/nova_pnab_-_portaria_no_2488_0.pdf
• ²⁰
  Polit FD, Cheryl TB. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489-97.
• ²¹
  Brasil. Ministério da Saúde; Conselho Nacional de Saúde. Resolução n. 466, de 12 de dezembro de 2012. Dispõe sobre diretrizes e normas regulamentadoras de pesquisas envolvendo seres humanos [Internet]. Brasília; 2012 [citado 2015 ago. 09]. Disponível em: http://bvsms.saude.gov.br/bvs/saudelegis/cns/2013/res0466_12_12_2012.html
  » http://bvsms.saude.gov.br/bvs/saudelegis/cns/2013/res0466_12_12_2012.html
• ²²
  Niquini RP, Bittencourt SA, Lacerda EMA, Saunders C, Leal MC. Avaliação da estrutura de sete unidades de saúde da família para a oferta da assistência nutricional no pré-natal no município do Rio de Janeiro, Brasil. Rev Bras Saúde Mater Infant [Internet]. 2010 [citado 2016 maio 14];10(1):61-8. Disponível em: http://www.scielo.br/pdf/rbsmi/v10s1/06.pdf
  » http://www.scielo.br/pdf/rbsmi/v10s1/06.pdf
• ²³
  Brasil. Ministério da Saúde. Avaliação normativa do Programa Saúde da Família no Brasil: monitoramento da implantação e funcionamento das equipes de saúde da família: 2001-2002 [Internet]. Brasília; 2004 [citado 2016 dez. 18]. Disponível em: http://189.28.128.100/dab/docs/publicacoes/geral/avaliacao_normativa_psf.pdf
  » http://189.28.128.100/dab/docs/publicacoes/geral/avaliacao_normativa_psf.pdf
• ²⁴
  Brasil. Ministério da Saúde. Guia prático do Programa de Saúde da Família [Internet]. Brasília; 2001 [citado 2016 set. 04]. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/partes/guia_psf1.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/partes/guia_psf1.pdf
• ²⁵
  Brasil. Ministério da Saúde. Política Nacional da Atenção Básica [Internet]. Brasília; 2012 [citado 2016 set. 04]. Disponível em: http://189.28.128.100/dab/docs/publicacoes/geral/pnab.pdf
  » http://189.28.128.100/dab/docs/publicacoes/geral/pnab.pdf
• ²⁶
  Roecker S, Nunes EFPA, Marcon SS. The educational work of nurses in the Family Health Strategy. Texto Contexto Enferm [Internet]. 2013 [cited 2016 Jan 06];22(1):157-65. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072013000100019
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072013000100019
• ²⁷
  Vieira RMN, Vieira MM, Ávila CRB, Pereira LD. Fonoaudiologia e saúde pública. São Paulo: Pró-fono; 2008.
• ²⁸
  Barbosa CP, Aires JB, Farias IYS, Linhares FMP, Griz SMS. Newborn and infant hearing health education for nursing professionals. Braz J Otorhinolaryngol [Internet]. 2013 [cited 2017 Sept 18];79(2):226-32. Available from: http://www.scielo.br/pdf/bjorl/v79n2/en_v79n2a16.pdf
  » http://www.scielo.br/pdf/bjorl/v79n2/en_v79n2a16.pdf

Publication Dates

• Publication in this collection
  11 Oct 2018
• Date of issue
  2018

History

• Received
  19 Sept 2017
• Accepted
  27 Feb 2018