Content and layout validation of indicators to assess the degree of implementation of the Hearing Conservation Program

Purpose : to validate indicators and parameters to construct an assessment instrument for the Hearing Conservation Program. Methods: methodological research to validate the content and layout of indicators to assess the Hearing Conservation Program, using the Delphi method, as well as the semantic validation of an assessment instrument for the program, in compliance with the procedures used by the European group DISABKIDS. A total of 20 speech-language-hearing therapists participated in the study, answering validation questionnaires. The answers were tabulated and analyzed considering descriptive statistical data, establishing the item content validity index, the scale content validity index, the percentage of absolute agreement, and the content analysis. Results: of the 64 items submitted for validation, two were not considered appropriate. The set of items was considered representative. The validated indicators and parameters provided the means to construct an assessment instrument of the degree of implementation of the Hearing Conservation Program, which proved to be semantically valid. Conclusion: the indicators were validated to meet and represent the functions of quality, control, and follow-up of the Hearing Conservation Program, aiding administrators to carry out their responsibilities and making it possible to construct evaluative instruments.


INTRODUCTION
The control of risk conditions and the improvement of work settings are central aspects of the actions aimed at occupational health promotion and protection. They identify and evaluate the risk conditions present in the work setting, characterizing the exposure, quantifying these conditions, implementing measures, and assessing them 1 .
Service Order 608, issued by the Instituto Nacional do Seguro Social (INSS -National Social Security Institute) on August 5, 1998, establishes the technical norms regarding sensorineural hearing loss due to continuous occupational exposure to high sound pressure levels and provides the means to implement the hearing conservation program (HCP), listing indispensable steps to maintain its effectiveness 2 .
The HCP is a dynamic and continuous process involving a set of actions aimed at noise control, audiometric monitoring, collective and personal protection, and health instruction, whose goal is to protect the workers' health by preventing the onset or worsening of noise-induced hearing loss (NIHL) in the workplace.
Despite the general guidelines to implement the HCP, the program can be conducted in different forms, revealing substantial differences in how the departments involved interpret and administer the legal and organizational requirements 3,4 . There are also difficulties resulting from the program implementation, besides the little assessment experience on the part of those involved 5 .
Assessing health programs in the work setting must be a priority in terms of occupational safety and administration 4 . Despite the recommendations and legislative endorsements, there are still few studies to be found in the national literature approaching the analysis and assessment of HCP in companies.
The apparent effectiveness of the HCP reflects gaps in knowledge about the individual aspects of the program and its effects on the hearing health results of the professionals involved 6 .
Every intervention can be understood as an organized action system whose objective, in a given time and context, is to correct a problematic situation 7 . As parts of this system, the structure, the people's practices (personal and collective), the action processes, the purposes, and the environment are components essential to an intervention. In this perspective, the assessment approaches must observe, measure, and define the priority information for the intervention to work [8][9][10] .
The essential aspects of an intervention may be represented by indicators that point out, describe, and characterize information, reflecting the care and attention patterns. Hence, identifying and establishing indicators is a crucial step to define the important and representative aspects and values in the intervention. Based on such understanding, the assessment and reference devices can be developed for the whole program.
Measuring and monitoring the indicators have many purposes -e.g., estimating the quality of the health care or intervention, making judgments, establishing priorities, supporting administrators and leaders, and improving the quality of the service 11,12 .
Validated indicators are expected to give the basis on which to construct assessment instruments that meet and represent quality, control, and follow-up functions of the components involved in the HCP. These are useful for the program and company administrators to monitor and assess the effectiveness and efficiency of the program.
The benefits of the assessment approaches make it possible to identify trends and improve the program, reflecting on the quality of the service, the efforts, and the expected results. Given the above, this research aimed to validate indicators and parameters to construct an HCP assessment instrument.

METHODS
The study was approved by the Research Ethics Committee of the Universidade Federal de Pernambuco (Federal University of Pernambuco), Brazil, under evaluation report number 1.978.729. It was developed in four stages, as described below. This is methodological research with a quantitative and qualitative approach, developed from June 2018 to May 2019.
The sample, comprising 20 speech-languagehearing therapists, was selected by convenience with the snowball technique. The key informants (n = 3) were selected based on their curriculum, which was consulted in the Lattes platform of the National Council for Scientific and Technological Development (CNPq, its Portuguese abbreviation). They were first contacted via e-mail, as registered in the platform, whereas the other participants were initially contacted via phone call or e-mail, as indicated by the key informants.

Stage 1: Establishing criteria, indicators, and parameters of the HCP
In the first stage of the research, the researchers surveyed the assessment indicators and parameters of the program, based on a Logical Operating Model of the HCP 13 . Also, the Manual of Guidelines and Parameters to implement the HCP was consulted 14 as a complementary document to survey the indicators. The surveyed indicators and parameters made up an analysis and judgment matrix (AJM), whose items were submitted for validation.

Stage 2: Content and layout validation of the indicators and parameters to assess the HCP
The content validation consists in evaluating whether a given concept represents the extent of the phenomenon of interest and whether each item encompasses that which it is meant to investigate 15,16 . The layout validation judges the relevance and adequacy of the form in which the items are presented 17 .
Ten speech-language-hearing therapists included in this stage comprised the panel of experts. Most of the participants were females (n = 9; 90%), mean age 42.6 years (SD = 10.95), with a specialization (90%) and a doctor's degree (10%), who worked in the private sector (90%) administering HCP (80%). Their professional experience with the program ranged from 3 to 25 years (mean = 11.1; SD = 8.03).
The AJM of the HCP allowed for the development of partially open-ended questionnaires to validate the content and layout of the items, structured in Google Forms ® (Appendix A). In each collection round, the questionnaire was accessed with a link to the electronic form, sent via e-mail to the participants.
The content and layout of the items were validated with the Delphi method, which consists of a series of rounds where a group makes judgments and comments on criteria and parameters of a given phenomenon to reach a consensus. The method requires confidentiality, the participation of experts, interactive rounds with feedback, and an effort to reach a consensus 18 .
At first, each item proposed in the AJM was assessed based on different criteria, with an adapted ordinal four-point Likert-type scale that categorized the items as (1) inadequate, (2) little adequate, (3) quite adequate, and (4) fully adequate. Using an ordinal four-point scale is recommended to avoid a neutral, ambivalent middle score 19 . The other phases in the Delphi method included administering questionnaires with clearer and more detailed formulations to make the study more understandable to those involved. Then, the comments, critiques, and opinions regarding the content and layout of the items were considered 18,20 . A schematic drawing of the application steps of the Delphi method is shown in Figure 1.
The criteria for content validity of the items considered their appropriateness, objectivity, simplicity, clarity, and precision. As for layout validity, the criteria encompassed presentation, clarity of the statements, readability, interpretation, and representativity. Also, some room was made available for the experts to make observations in the assessment of the items.
Concerning the verification parameters of the indicators proposed in the AJM, the suggested scores for each indicator ranged from one (1.0) to three (3.0) points. They were then evaluated according to their appropriateness.
The data from the validation process were tabulated in Microsoft Office Excel 2010 and treated quantitatively based on descriptive statistical analysis. The qualitative analyses were conducted with content analysis 21 , whose stages comprised a pre-analysis of the experts' observations and suggestions, followed by an exploration of the material and treatment of the results, establishing and organizing the recurrent theme categories.
The results of the content validation process, specifically the assessment of the appropriateness of the items and their scores, were analyzed based on the item content validity index (I-CVI) and scale content validity index (S-CVI). The condition established to validate each item was I-CVI and S-CVI values equal or superior to 0.80, categorizing them as adequate 22 . The items that scored "1" or "2" were revised in the subsequent phases of the Delphi method and submitted to new judgment by the judges until a consensus was reached.
The content and layout validity criteria were analyzed with the percentage of absolute agreement, obtained by summing the items that were assessed positively and dividing it by the total of answers. The criteria that achieved 80% or more were considered valid.  Assessment of the appropriateness of the indicator;  Assessment of the appropriateness of the score proposed;  Assessment of content and layout criteria of the items.
2 not validated items. 20 items proposed for revision and consensus on one or more content and/or layout validation criteria; 12 validated items were given comments and suggestions to be included in the second round.

STAGE 3: CONSTRUCTION OF THE HCP ASSESSMENT INSTRUMENT
The validated items of the AJM of the HCP were the basis for the construction of the first version of the program assessment instrument as a structured questionnaire. The normative assessment instrument makes it possible to judge the assessment items in contrast with the expected verification parameters (norm/score standard), whose result enables the program's degree of implementation to be estimated with different cutoff scores.
The manner how the programs are implemented may be as important as the results, especially when they involve the participation of various professionals and their integration with the administrators in the implementation process 23 . Considering the importance and complexity of the processes implemented in the HCP, these components' score was multiplied by six, whereas the structure components' score was multiplied by four for the analysis. Thus, the total degree of implementation was established by calculating the weighted mean of the answers obtained in the application of the instrument, with the formula presented in Figure 2, whose result was presented in percentages. The value judgment of the degree of implementation considered four cutoff scores (< 25.0% -not implemented; from 25.1% to 50.0% -incipient implementation; from 50.1% to 75.0% -partially implemented; and > 75.1% -implemented).
Total DI � � �4 ∑ � � 6 ∑ � � � 10 �4 ∑ � � 6 ∑ � � � 10 � � 100 Caption: DI = degree of implementation; Σ S¹ = Sum of the scores obtained in the indicators that made up the structure dimension; Σ P¹ = Sum of the scores obtained in the indicators that made up the process dimension; Σ S² = Sum of the scores expected for the indicators that made up the structure dimension; Σ P² = Sum of the scores expected for the indicators that made up the process dimension.

Stage 4: Semantic validation of the HCP assessment instrument
The semantic validation consists in the theoretical analysis of the items proposed in the instrument, approaching the understanding and acceptance of the terms, the relevance of the items, the existence of difficulties, and the possible need for adjustments 12 .
The data from the semantic validation of the HCP assessment instrument were collected by administering the initial version of the instrument itself to the participants, as well as two validation questionnaires based on the research of the European group DISABKIDS 24 . In this approach, a general impression questionnaire was proposed (general semantic validation) along with a specific semantic validation questionnaire for the HCP assessment instrument.
The analyses were conducted with the percentage of absolute agreement, considering valid the criteria with values equal to or higher than 80%. The experts' observations and suggestions underwent a content analysis 21 , whose data were analyzed and categorized, justifying the necessary readjustments in the instrument.

RESULTS
Considering the HCP Logical Operating Model, 64 indicators were defined, 28 items referring to the structure component, and 36, to the processes, which were the basis to create the first version of the AJM of the HCP.
The AJM was developed considering the HCP dimensions, criteria, indicators, and standard (norms) expected for each indicator and the respective verification parameters (expected score and cutoffs). The criteria represent one or more aspects of the action or service and are represented by indicators, which work as attributes used to measure the components of a program 25 . The norms or standard is the conceivable value of the criterion.
The response rates obtained in the content and layout validation process were 47.6% (10/21) in the first round of the Delphi method (Stage 2), 90% (9/10) in the second round, and 66.6% (6/9) in the third and last round. Everyone's answers were considered for analysis, even of those who had stopped participating in previous rounds -hence, 10 participants were considered for this stage.
A difficulty commonly found in using this methodology is that some participants abandon it from one round to the other, either because the process takes too long or because it requires too much time 26 . However, since the results are analyzed by the researchers in between each round of questionnaires, it is important that all the opinions -including the minority ones of participants who left it -be also shared and reported. Thus, the panel of experts has access to them and can include them in their considerations and arguments 27 .
Two structure indicators out of the items proposed in the first validation round were not considered appropriate -i.e., these indicators were not considered relevant, as they did not meet the intended purpose. Item 20, which refers to the "Existence of an Awareness Statement", was considered appropriate by only 60% (I-CVI = 0.60) of the experts. Item 22, which refers to the "Existence of a duplicate of the Occupational Health Certificate (OHC)" was considered appropriate by 70% (I-CVI = 0.70) of the experts.
Altogether, the items obtained a representative agreement level (S-CVI = 0.98), and their amplitude and balance were considered adequate. The score of all the items was considered appropriate (S-CVI = 0.93).
Some items (n = 6), whose appropriateness remained valid, did not reach a considerable agreement in one or more content assessment criteria (Table 1), while others (n = 14) did not reach a considerable agreement in one or more layout assessment criteria ( Table 2). In other cases, despite having an adequate content, layout, and scores, the items (n = 12) received comments and/or suggestions that were included for the second consensus round, leading to revisions in its presentation.
In the second consensus round, most of the items proposed (n = 23) were revised -and then their content and layout reached a consensus -aiming at greater objectivity, simplicity, clarity, and precision of the statements, as well as improvements in their presentation, readability, clear presentation, interpretation, and representativity. In this round, new suggestions came up regarding some items (n = 9), which were presented for consensus in the third round.
After the third and last validation round, all the suggestions reached a consensus. Altogether, 62 items were validated, considering 26 indicators related to structure and 36, to processes. A representation of the rounds is given in Annex 1.
The AJM provided valid means to develop the first version of the HCP assessment instrument to estimate the degree of implementation of the program. It is a structured, standardized questionnaire, whose structure includes information on the company, the assessment, presentation of the instrument with instructions to fill it in, two sets of questions regarding the items to be assessed (resources and activities), a sheet for interpreting and analyzing the results to establish the degree of implementation of the HCP, and in the end a sheet to present the results and indicate necessary improvements.
There were 26 assessment questions in the first set, approaching the resources necessary for an efficient implementation of the HCP. These work as the basis for the actions to be implemented. The second set comprised 36 questions regarding the processes (actions) to be implemented, that is, the activities that need to be carried out in the program. Each question has a description of the expected standard answer, setting a parameter for the assessment. Considering the appropriateness validity of the scores proposed, it was established that the expected summed score of the set of structure items of the HCP would be 55.0 points, while that of the process items would be 87.0 points; hence, the overall expected score was established at 142 points. The existence of resources and the implementation of processes according to the established parameters must be given the total expected score. The absence or non-presentation of resources, as well as the non-implementation or the partial implementation of processes, are not given any score.
The scores obtained in each set are analyzed according to the instructions given in the sheet for the interpretation and analysis of results, present in the instrument. The analysis of the results must consider the scores obtained in relation to the one expected in each set of the assessment. The total degree of implementation is then presented in percentage, resulting from the calculation of the weighted mean of the results obtained in relation to the expected in each set. The degree of implementation is classified based on the cutoff scores distributed into four tiers 25 , as described below: a) Not implemented: < 25% b) Incipient implementation: 25.1% to 50% c) Partially implemented: 50.1% to 75.0% d) Implemented: >75.1% In the semantic validation of the HCP assessment instrument (Stage 4), all the participants (n = 10) considered the instrument good (20%) or very good (80%). The participants assessed the questions present in the instrument mostly as "easy to understand" (90%), while there were "some difficulty" (10%). The answer options were assessed as "easy to understand" (90%) or "with some difficulties" (10%). Everyone considered the questions present in the instrument very important to assess the HCP. Most of them had nothing to change or add to the questionnaire (90%), and they felt comfortable answering all the questions included in the assessment instrument. Some suggestions were made, leading the instrument to be revised. The analysis categories in consideration were in the field of information on the company, instructions to fill in the instrument, and the sheet of interpretation and analysis of the results. The final degree of implementation of the HCP is shown in Annexe 1.

DISCUSSION
Considering the results obtained, it can be pointed out that, in general, the HCP process items easily reached a consensus among the experts. Only two of the items regarding this component were not considered valid and the suggestions and observations made while the other items reached a consensus dealt with the form of presentation of the indicator, suggesting clearer and more objective statements.
There are, in the literature, many suggestions on the minimally necessary stages to effectively implement the HCP activities. Among them, there is the practical guide proposed by NIOSH 28 , a pioneer and reference agency in the development of research and guidelines involving the HCP and, more recently in the national context, the Manual of Guidelines and Minimal Parameters to Develop and Administer the Hearing Conservation Program, issued by FUNDACENTRO 14 . Based on these, the actions developed in the HCP are admitted and implemented as a consensus among administrators. However, the necessary structures for an efficient implementation of the program -considering the physical, human, and organizational resourcesare not clearly defined in the guides. In this context, the items proposed regarding the HCP structure components raised many considerations by the experts, and it was necessary to reformulate them.
The content, layout, and form of presentation of the item "Existence of an Awareness Statement" were not considered clear (percentage of absolute agreement = 70%), neither was it considered appropriate (I-CVI = 0.60). The hearing loss awareness statement is a document in which the employee, after reading and signing it, states that they have had auditory rest for 14 hours and then undergone audiometric examination and that they have been informed of the result in case of hearing loss.
Such a practice is not provided by law. Nevertheless, it has been implemented in companies to have it registered as a means of protection because of the employees' mandatory 14-hour auditory rest before undergoing audiometric examinations and the companies' obligation to inform the result to the employees, especially when there is a hearing loss.
The law provides the auditory rest before undergoing initial and/or reference audiometric examinations 2,29 . However, it is unclear whether the auditory rest is mandatory before sequential examinations unless its result is suggestive of triggering NIHL. In this case, the audiometry is defined as either reference or sequential, thus requiring a new evaluation in light of the guidelines proposed by the Regulatory Norm no. 7, considering the auditory rest 29 .
The time of auditory rest -which is recommended in Service Order 608 -is necessary for the auditory effects caused by the high sound pressure levels (such as a temporary threshold shift) not to interfere with the result of the exam 2 . In practical terms, considering the information in the audiological anamnesis, in case the employee has been exposed to high sound pressure levels a little before undergoing the examination and there are changes in the auditory thresholds in relation to the initial/reference examination, they are asked to return 24 hours later for a new examination.
The experts understand that "it is up to the company to ensure that the employee has time for the auditory rest". The Regulatory Norm no. 7 establishes that "the employee will have an auditory rest for at least 14 hours before undergoing the examination" 29 -which is one of the basic principles and procedures to perform an audiometric examination. On the other hand, Service Order 608 does not explicitly point out this responsibility. Hence, the law does not clearly indicate whose responsibility it is to ensure the auditory rest, neither is there instruction on how to effectively control such time.
Also, the experts' question "the feasibility of a 14-hour auditory rest for sequential examination because it implies in releasing the employee from their work duties".
This deadlock leads companies to use strategies to meet the guidelines and better administer the diagnoses. These include the implementation of educative 30 and organizational measures, scheduling the work in accordance with the planned examinations. Also, the employees can be informed and motivated to use the necessary preventive and auditory rest strategies before undergoing an audiometric examination.
Another aspect brought up by the experts was that "the employee signs the result of the examination", thus stating their awareness. The Regulatory Norm no. 7 recommends that reference auditory examinations be established, especially when hearing losses are present or triggered, comparing them with sequential ones for monitoring purposes 29 . Likewise, Service Order 608 instructs that the employees must be informed of the results of each examination they are submitted to and receive a copy of them -although the diagnosis of a possible hearing loss does not disqualify the employee to perform their work duties 2 . Nonetheless, the employee's signature on the audiological examination does not ensure that such information was actually passed on to them -this practice only registers and proves that the examination was made.
Also, the experts did not consider the item that refers to the "Existence of a duplicate of the Occupational Health Certificate (OHC)" as an HCP indicator (I-CVI = 0.70). The OHC, a reference document of the Occupational Health Medical Control Program (OHMCP), is a certificate issued by the occupational physician. Besides stating they are apt to work, the certificate points out the specific occupational hazards and the medical procedures used, including complementary examinations 29 . The experts argued that "the OHC indicates whether the employee is apt to perform their duties in the company" and that it "evaluates whether the employee's health is in accordance with the risk exposure in the activities they must perform" and helps "the occupational safety administration". Hence, it is not relevant to the HCP.
The law points out that having a hearing loss induced by high sound pressure levels does not in and of itself indicate the person is unfit for work. Each case must be analyzed separately, considering the auditory requirements of the work or position 29 . Hence, the one responsible for defining whether the employee is fit for work when they are with suspicion for hearing loss induced by high sound pressure levels is each company's OHMCP coordinating physician. They must consider the assessment of the type and degree of hearing loss presented by the employee, the duties they are to perform, the frequent audiological control, and educative actions 31 .
It should be highlighted that the OHC must state the risks the employee will be exposed to while performing their work activities. This is one of the indicators for occupational safety administration, revealing the need for audiological administration and monitoring of employees exposed to high sound pressure levels 29 .
The definition of the structure indicators related to human resources regarding each one's responsibilities to perform different actions led the experts to put many propositions forth. The human resources include the people involved in implementing the interventionan essential aspect for the services to be adequately carried out.
The HCP, which is an interdisciplinary program with cooperation between sectors, is meant to relate different processes administered by different people -which poses a challenge for the program's administration. Considering the processes involved in the implementation of the program, there supposedly is a need for the company's professionals in the fields of health, safety, management, human resources (the company's owner or personnel administrator), and, most of all, the employees who will implement the HCP 2,14 . Including other professionals will depend on the complexity of the program, considering the company's structure and context.
Concerning the "existence of Specialized Safety Engineering and Occupational Medicine Services (SESMT, its Portuguese abbreviation), person or team capable of developing engineering projects and measures" and the "participation in the implementation in the engineering measures", there were different arguments about what professional would be able to implement such services and whether they should be developed by only one specialized professional or by a team, considering the occupational physician and the speech-language-hearing therapist. As for the law, Regulatory Norm no. 9 advocates that the actions of the Environmental Risk Prevention Program (ERPP) must be carried out by the SESMT, or by a person or team that, according to the employer, can develop what is provided in this Regulatory Norm 32 . Although it does not clearly specify what professionals have the responsibility, it can be said that having a specialized occupational safety engineer in the team that implements the HCP is utterly important. After all, this is the technically qualified professional to develop engineering measure projects. Including other necessary professionals and qualifications will depend on the comprehensiveness of the preventive measures proposed.
Also, regarding the person or team that can develop the HCP, there was no consensus at first concerning what professionals can implement and administrate the program, as such responsibilities are not clear in the law. The experts suggested to include, besides the speech-language-hearing therapist and occupational physician, the occupational safety engineer as one of the professionals who can administrate the HCP.
Considering the multidisciplinarity of the HCP, even before defining its administrator, it is important that the responsibilities be well-defined and delegated to technically capable professionals, considering the specificities of each stage to be implemented, its needs, and necessary competencies 14 . In this regard, a consensus was reached in terms of considering the existence of a technically capable professional or team to develop the program.
Along this line, there were different standpoints on the professional specialty responsible for carrying out the audiometric examinations. The law provides that they must be conducted only by a qualified professional (i.e., a speech-language-hearing therapist or a physician), as stated in resolutions of their professional councils, whereas no other specialty is pointed out 2,29 . Hence, some experts argued that "the audiometry can be conducted by any medical professional, as long as they are specialized in audiology". Others counterargued that, despite the regulatory law mentioning the physician, "few professionals in the field have the experience and skill to conduct the audiometric examinations; so, the speech-language-hearing therapist is the one who should do it". After the following consensus rounds, the "existence of a professional specialized in audiology to conduct the audiological examinations" was established as a parameter for this indicator.
Concerning the "Inspection of personal protective equipment (PPE) use", one of the experts suggested that this role "could be the responsibility of the occupational safety technician". The suggestion was taken to the other experts for consensus, and they agreed that it is important to evaluate the HCP administrator's cooperation with the person or team responsible for it and the other people of the program involved in inspecting the employees' PPE use.
The employees' PPE use is a complex habit, influenced by various factors, including intrapersonal (such as discomfort or interference with the communication), interpersonal (including the relationship with workmates, support personnel, and supervisors), and organizational factors (involving organization rules, spreading knowledge and information, and so on) 33 . It is highly important to observe these factors associated with the aspects of health education and commitment of all those involved in the HCP in order to develop safe behaviors, particularly regarding PPE use.
To establish assessment parameters, aspects considered important and priority were surveyed, as they would reflect the quality of what is being assessed. In this sense, some structure indicators -as in the case of reports -had a set of variables to be observed in the assessment, specifically the resources related to the "Existence of a risk analysis report", "Existence of an ERPP planning report", "Existence of an OHMCP report", "Existence of a PPE attenuation review", "Existence of an HCP implementation report", and "Existence of an individual clinical record". Such items were proposed for a consensus in the following rounds, aiming at a better definition and clarity of the parameters to be assessed. It was generally agreed that it is important to specify the variables that would reflect actually important information for the HCP. Thus, the parameters were reformulated according to the experts' suggestion and agreement.
Overall, the structure of the instrument was wellassessed by everyone regarding the fields of information on the company and on the assessment, the instructions to fill in the instrument, the sets of structure and process assessment, the sheet for the analysis and interpretation of answers, and the sheet of results, pointing out the clear description and understandability.
The semantic validity of the HCP assessment instrument reflected a satisfactory evaluation on the part of the consulted professionals. The form of presentation and listing of resources and processes to be assessed were deemed "well-written and easy to understand", as pointed out by one of the professionals. They considered that "the listing and description of the necessary resources to implement the HCP are great", and that "the sequence favors the identification of the actions to be implemented".
The description of the expected parameters for each item was considered positive, as it "makes the assessor's job easier, making it clearer what they have to observe". The checklists of the actions to be implemented and assessed in the HCP are widely used since the publication by NIOSH 28 , evidencing the importance of checking separately the quality and the full carrying out of each implementation stage of the program in the periodic audits. Research also points out that using a predeveloped assessment guide makes it easier to verify the functioning of the HCP and the detection of flaws in the program 34 .
The professionals commented on the objective and functionality of the instrument, pointing out that the material "will enable the researcher to know the companies that have an effective HCP and the ones that still need to implement it". The degree of implementation estimate is an alternative to assess the program implementation process. It gives the basis for administrators, assessors, and researchers to both solve possible problems in the operationalization of the intervention and to formulate recommendations aimed at improving the program. Such an estimate can be applied before, during, or after implementing the HCP 35 .
Also, the participants highlighted the importance of the assessment instrument to monitor the program, stating that "it is good for the companies as well, for them to see what they are doing wrong". The HCP assessment by audits aims to determine, among other things, whether the actions implemented in the program were effective to meet the objectives outlined for occupational health and safety, and whether the company complies with the existing legal requirements 36 , thus revealing possible mistakes and deviations from the norm. However, other evaluative surveys must complement the results obtained from the audits 37 . This makes it possible to understand the current condition of the program in relation to different aspects involved in the intervention -which may both influence the degree of implementation and be influenced by itsuch as the contextual aspects (political and structural), the expected results, and their impacts. The degree of implementation then becomes an important variable to observe and analyze these conditions. The validation of this instrument is necessary to obtain reliable measures (based on the application of the instrument in different contexts) and statistical analysis of stability, internal consistency, and equivalence. In this perspective, other psychometric properties of the instrument can be tested to verify the precision of the data and measures, which must be valid and interpretable to assess the HCP.

CONCLUSION
The validity of assessment indicators and parameters for the HCP proves to be important in the planning, analysis, and monitoring of the program. Hence, they provide the means to construct the evaluative instruments that encompass the operational components of the intervention, namely, the structure, processes, and expected results.
The instrument to estimate the degree of implementation of the HCP aims to reveal the current condition of the intervention, considering criteria, indicators, and parameters validated by experts. It can be self-applied or directed to the administrator of the program before, during, or after its implementation.  • The instrument aims to estimate the degree of implementation of the HCP -i.e., the condition/level in which the program was/ is being implemented. • The instrument is divided into two sets. In the first assessment set, there are 26 questions on the resources necessary to effectively implement the Hearing Conservation Program. These are the basis for the actions that will be implemented. The second set comprises 36 questions on the processes (actions) to be implemented -i.e., the activities that need to be developed in the program. • Each question has a description of the expected answer, setting an assessment parameter. In the assessment of resources, the maximum expected score is 55.0 points. As for the processes, the maximum expected score is 87.0 points. The total expected score for the instrument is 142 points, which would reflect a fully implemented Hearing Conservation Program. • The existence of resources and the implementation of processes following the parameters specified in the instrument must be given the total expected score for each question. The absence or non-presentation of the resources, as well as the nonimplementation or partial implementation of processes, are not given any score.

Is there a person or position in the company responsible for ensuring the resources and administrative measures to carry out the Hearing Conservation Program?
Existence of (1) Specialized Safety Engineering and Occupational Medicine Services (SESMT) and/or (2) at least one department, person, or representative directly related to the company.

Is there in the company a Specialized Safety Engineering and Occupational Medicine Service (SESMT), or person or team that can develop engineering projects and measures?
Existence of an engineer specialized in occupational safety or an interdisciplinary team that includes this professional 2.0

Does the company have an occupational physician responsible for the occupational health medical control program?
Existence of an occupational physician 3.0

Does the company have a professional or team technically qualified to develop the Hearing Conservation Program?
Existence of an occupational physician and/or speechlanguage-hearing therapist and/or occupational safety engineer 3.0

Does the company have a risk analysis report?
Existence of an annual risk analysis report, presenting at least general data of assessment and description of the exposure risks and conditions.

Does the company have an Environmental Risk Prevention Program planning report?
Existence of an annual report or whenever there is any change in the risk analysis report, presenting the (1) definition of goals, priorities, and schedule; (2) action strategy and methodology (according to the hierarchy of the control measures); (3) form of the registry, maintenance, and communication of the data; (4) Periodicity and form of assessment of the development.

Does the company have a report on the occupational health medical control program?
Existence of an annual report on the occupational health medical control program, presenting data from the administration of audiological diagnoses, with statistics of the results of nosologic diagnoses and evolution diagnoses by departments of the company.

3.0
Does the company have an implementation report of the Hearing Conservation Program?  (7) assessment of the program, in case it has been implemented before.

3.0
Is there any proof that the audiometric examination is performed in an audiometric booth or an acoustically treated environment?
Existence of a review of the sound booth or acoustically treated environment to perform the audiometric examinations, in compliance with ISO 8253-1.

Does the company provide informational material in its facilities?
Existence of bulletin boards and/o risk maps in the facilities of the company, presenting the risks to hearing health, indicating areas with high sound pressure levels.

Does the company have an attenuation review of the personal protective equipment (PPE) used by the employees?
Existence of an annual attenuation review of the personal protective equipment used by the employees, presenting the techniques used, the equipment analyzed, and the results obtained.

Does the company have personal protective equipment adequate to the risk of each activity?
Existence of hearing protection device(s), which can be circumaural hearing protection, insert hearing protection, or semi-auricular hearing protection.

Does the company have a Certificate of Approval of the personal protective equipment used by the employees?
Existence of a certificate of approval of the PPE, issued by the national department responsible for occupational health and safety issues in the Ministry of Labor and Employment.

2.0
Is there a specialized professional to carry out the audiological examinations?
Existence of a professional specialized in audiology to carry out the audiological examinations.

2.0
Has the professional proved their qualification to carry out the audiological examinations?
Existence of a receipt from the pertaining professional council proving the qualification as a physician or speechlanguage-hearing therapist who carries out the audiometric examinations.

3.0
Is there any proof that the audiometric examination is carried out with duly calibrated measuring equipment?

EVALUATIVE QUESTION EXPECTED STANDARD/NORM EXPECTED SCORE SCORE OBTAINED
Have the responsibilities, goals, and priorities of the Hearing Conservation Program been defined or are they being defined?
Responsibilities, goals, and priorities in the period encompassing the implementation of the Hearing Conservation Program and/or its annual monitoring.

3.0
Is a preliminary assessment of noise exposure at the company conducted/has it been conducted?
Characterization of the risk exposure, applied to all the operational and habitual conditions of the employees when performing their duties, either annually or whenever there is a change in the environment.

3.0
Have homogeneous groups with similar exposure characteristics been identified/are they identified?
Identification of homogeneous groups with similar exposure characteristics, considering all those exposed in the group in question or covering one or more employees whose situation corresponds to the "typical" exposure of each group in question.

2.0
Is there/has there been an analysis of (co)existence of other risk agents (chemical, physical, and biological)?
Recognition and assessment of the employees' exposure to chemical, physical, and/or biological agents that might trigger an occupational hearing loss.

2.0
Is the noise exposure in the company quantitatively assessed/has it been assessed?
Obtention of an estimate of the employees' exposure, according to (1) the established daily dose of noise exposure, considering 100% as the limit of daily occupational continuous or intermittent noise exposure and 50% as the daily dose for the action level of occupational noise exposure; (2) the established exposure level and normal exposure level, considering his one equal to 80 dB(A); (3) the established correlation between maximum admissible peak levels and the number of impacts occurred during the workday, considering the peak level of 140 dB(Lin) as the tolerance limit of impact noise.

3.0
Have the action strategies and methodologies of the Hearing Conservation Program been established/are they being established?
Established action strategies and methodologies and the annual schedule or the one regarding the time encompassed in the current implementation of the Hearing Conservation Program.

3.0
Is the annual planning of the Hearing Conservation Program presented to and discussed with the departments involved in it/has it been presented and discussed?

2.0
Is the personal protective equipment properly selected/ has it been properly selected?
Selection of hearing protection devices technically adequate to (1) the characteristics of the environment and activity; (2) characteristics of the user; (3) characteristics of hearing protection; (4) required noise reduction level; (5) noise reduction level of the protection; (6) exposure level with the protection; (7) annual personal attenuation review, or whenever the model or size is changed.

2.0
Is the personal protective equipment supplied to the employees and replaced/has it been supplied and replaced?
Supply of hearing protection devices approved by the national department responsible for occupational safety and health and immediate replacement of the equipment when it is damaged or lost.

2.0
Is there/has there been an inspection of the use of the personal protective equipment on the part of the employees?
Inspect the use of personal protective equipment on the part of the employees, making use of (1) monitoring protocol of the use of hearing protection and the necessary periodicity, and (2) communicate to the Ministry of Labor and Employment any irregularity observed.

2.0
Have the employees who are entitled to a hazard pay or special retirement due to noise exposure defined/are they defined?
Aid the team to define the employees who must receive a hazard pay or special retirement due to noise exposure and refer them to Social Security.

EVALUATIVE QUESTION EXPECTED STANDARD/NORM EXPECTED SCORE SCORE OBTAINED
Are verification and control procedures regarding the functioning of the audiometer periodically conducted/ have they been conducted?
(1) Submit the audiometer to annual electroacoustic verification, in compliance with norm ISO 8253-1; (2) Perform biological verification of the audiometer(s) before the audiometric examinations.

3.0
Are the employees submitted to occupational anamnesis/ have they been submitted to it?
Clinical occupational anamnesis conducted in the audiometric assessment, in all types of examinations (preemployment, sequential, post-leave of absence, and postemployment examinations).

2.0
Are the employees' ears examined (meatoscopy)/have they been examined?
Examination of the external acoustic meatus of both ears by the person responsible for the audiometric examination, (1) when performing an audiometric assessment, in all types of examinations (pre-employment, sequential, post-leave of absence, and post-employment examinations) and (2) writing the findings in the audiometric examination form.

2.0
Are all the employees submitted to the pre-employment audiometric examination/have they been submitted to it?
Submitting the employees to pre-employment audiometric examination.

3.0
Are the audiometric examinations classified as a reference?
Establishing the reference audiometric examination when the employee does not have one or whenever an occupational hearing loss is triggered or worsened.

3.0
Are sequential audiometric examinations conducted/have they been conducted?
Conducting sequential audiometric examinations (1) in the 6th (sixth) month after employment or based on the reference audiometric examination; (2) annually, beginning in the 6th (sixth) month after employment.

3.0
Are post-employment audiometric examinations conducted/have they been conducted?
Conducting audiometric examination when the employee is dismissed, except for the situations provided in the Regulatory Norm no. 7.

3.0
Is speech audiometry conducted/has it been conducted?

2.0
Is the evolution of hearing loss analyzed and diagnosed/ has it been analyzed and diagnosed?
Referring the employee to the physician who coordinates the Occupational Health Medical Control Program to (1) analyze the evolution and define the evolution diagnosis of all the examinations performed, classifying the results as "normal hearing", "occupational hearing loss", or "nonoccupational hearing loss"; (2) characterize the causal nexus between auditory damages and environmental risks, to guide the environmental control measures.

3.0
Are the results coming from the audiological diagnosis administration registered/have they been registered?
Including the cases suggestive of hearing loss induced by high sound pressure levels in the annual report of the Occupational Health Medical Control Program.

2.0
Are accidents at work communicated/have they been communicated?
Communicating the accidents at work suffered by employees with occupational hearing loss.

3.0
Are the employees given a duplicate of the examinations and certificates/have they been given them?
Making available (1) duplicates of audiometric examinations and (2) occupational health certificates to all the employees.

2.0
Are the records coming from the implementation of the program being administered and saved?
Maintaining, for at least 20 (twenty) years after the employee has been dismissed, the records of the results of the audiometric examinations, environmental assessments, and measures used for collective protection.

2.0
Are there/have there been training programs, debates, commissions, participation in events, and/or other appropriate forms involving the effects on health caused by the exposure to high sound pressure levels to all those involved in the Hearing Conservation Program?
Having carried out at least one of the actions in the previous 12 (twelve) months, involving the effects on health caused by the exposure to high sound pressure levels.

3.0
Are the employees trained about the implementation of collective and individual measures/have they been trained?
Having carried out at least one of the actions in the previous 12 (twelve) months, involving the procedures that ensure its effectiveness with hearing protection adjustment test, maintenance, replacement, hygiene, and information procedures regarding the possibly limited protection they provide.

3.0
Are the employees' suggestions welcomed/have they been welcomed?
Welcoming and registering the employees' suggestions brought up in meetings and debates.

EVALUATIVE QUESTION EXPECTED STANDARD/NORM EXPECTED SCORE SCORE OBTAINED
Are quantitative analyses of prevalence and incidence of the results of the audiological examinations carried out/ have they been carried out?
Annually presenting statistics of the results of nosologic diagnoses (prevalence: normal hearing, occupational hearing loss, or non-occupational hearing loss) and evolution diagnoses (incidence: triggering or worsening of occupational hearing loss).

Is the employees' degree of satisfaction assessed/has it been assessed?
Annually assessing the employees' degree of satisfaction, based on their opinions regarding the implementation of the Hearing Conservation Program.

Is the degree of implementation of the Hearing Conservation Program being estimated with audits/has it been estimated?
Measuring the degree of implementation of the components of the Hearing Conservation Program systematically before, during, or after implementing the program, at least once a year.

3.0
Are the contextual factors that can influence the implementation of the Hearing Conservation Program being considered and analyzed/have they been considered and analyzed?
Conducting overall and contextual analyses, identifying the favorable or unfavorable factors to the ideal implementation of the Hearing Conservation Program in the company, relating them to the results of the audits.

FORMULA TO CALCULATE THE WEIGHTED MEAN OF THE ANSWERS TO OBTAIN THE DEGREE OF IMPLEMENTATION OF THE HEARING CONSERVATION PROGRAM
Total DI � � �4 ∑ � � 6 ∑ � � � 10 �4 ∑ � � 6 ∑ � � � 10 � � 100 Caption: Σ S¹ = Sum of the scores obtained in the indicators that made up the structure dimension. Σ P¹ = Sum of the scores obtained in the indicators that made up the process dimension. Σ S² = Sum of the scores expected for the indicators that made up the structure dimension. Σ P² = Sum of the scores expected for the indicators that made up the process dimensio. Observation: • The formula above represents the calculation of the weighted mean of the answers, considering the score obtained with the application of the instrument in relation to the total expected score. The items that make up the structure (resources) are multiplied by 4 (four), while the items that make up the processes (activities) are multiplied by 6 (six). • The result obtained with the formula is presented in percentage.