Content and layout validation of indicators to assess the degree of implementation of the Hearing Conservation Program

ABSTRACT

Purpose:

to validate indicators and parameters to construct an assessment instrument for the Hearing Conservation Program.

Methods:

methodological research to validate the content and layout of indicators to assess the Hearing Conservation Program, using the Delphi method, as well as the semantic validation of an assessment instrument for the program, in compliance with the procedures used by the European group DISABKIDS. A total of 20 speech-language-hearing therapists participated in the study, answering validation questionnaires. The answers were tabulated and analyzed considering descriptive statistical data, establishing the item content validity index, the scale content validity index, the percentage of absolute agreement, and the content analysis.

Results:

of the 64 items submitted for validation, two were not considered appropriate. The set of items was considered representative. The validated indicators and parameters provided the means to construct an assessment instrument of the degree of implementation of the Hearing Conservation Program, which proved to be semantically valid.

Conclusion:

the indicators were validated to meet and represent the functions of quality, control, and follow-up of the Hearing Conservation Program, aiding administrators to carry out their responsibilities and making it possible to construct evaluative instruments.

Keywords:
Validation Studies; Program Evaluation; Noise-Induced Hearing Loss

RESUMO

Objetivo:

validar indicadores e parâmetros para a construção de um instrumento de avaliação do Programa de Conservação Auditiva.

Métodos:

pesquisa metodológica de validação de conteúdo e aparência de indicadores para avaliação do Programa de Conservação Auditiva mediante aplicação do método Delphi, bem como a validação semântica de um instrumento de avaliação do programa seguindo os procedimentos adotados pelo grupo europeu DISABKIDS. Participaram do estudo 20 fonoaudiólogos, os quais responderam a questionários de validação. As respostas foram tabuladas e analisadas considerando dados estatísticos descritivos, pelo estabelecimento dos índices de validade de conteúdo por Item (I- IVC), de validade de conteúdo por nível de escala (S-IVC), de nível percentual de concordância absoluta e Análise de Conteúdo.

Resultados:

dos 64 itens submetidos à validação, dois não foram considerados pertinentes. O conjunto de itens foi considerado representativo. Os indicadores e parâmetros validados subsidiaram a construção de um instrumento para avaliação do grau de implantação do Programa de Conservação Auditiva, o qual se apresentou válido semanticamente.

Conclusão:

os indicadores foram validados de modo a atender e representar as funções de qualidade, controle e acompanhamento do Programa de Conservação Auditiva, auxiliando os gestores no exercer de suas funções e subsidiando a construção de instrumentos avaliativos.

Descritores:
Estudos de Validação; Avaliação de Programa; Perda Auditiva Induzida por Ruído

Introduction

The control of risk conditions and the improvement of work settings are central aspects of the actions aimed at occupational health promotion and protection. They identify and evaluate the risk conditions present in the work setting, characterizing the exposure, quantifying these conditions, implementing measures, and assessing them¹.

Service Order 608, issued by the Instituto Nacional do Seguro Social (INSS - National Social Security Institute) on August 5, 1998, establishes the technical norms regarding sensorineural hearing loss due to continuous occupational exposure to high sound pressure levels and provides the means to implement the hearing conservation program (HCP), listing indispensable steps to maintain its effectiveness².

The HCP is a dynamic and continuous process involving a set of actions aimed at noise control, audiometric monitoring, collective and personal protection, and health instruction, whose goal is to protect the workers’ health by preventing the onset or worsening of noise-induced hearing loss (NIHL) in the workplace.

Despite the general guidelines to implement the HCP, the program can be conducted in different forms, revealing substantial differences in how the departments involved interpret and administer the legal and organizational requirements^3,4. There are also difficulties resulting from the program implementation, besides the little assessment experience on the part of those involved⁵.

Assessing health programs in the work setting must be a priority in terms of occupational safety and administration⁴. Despite the recommendations and legislative endorsements, there are still few studies to be found in the national literature approaching the analysis and assessment of HCP in companies.

The apparent effectiveness of the HCP reflects gaps in knowledge about the individual aspects of the program and its effects on the hearing health results of the professionals involved⁶.

Every intervention can be understood as an organized action system whose objective, in a given time and context, is to correct a problematic situation⁷. As parts of this system, the structure, the people’s practices (personal and collective), the action processes, the purposes, and the environment are components essential to an intervention. In this perspective, the assessment approaches must observe, measure, and define the priority information for the intervention to work^8-10.

The essential aspects of an intervention may be represented by indicators that point out, describe, and characterize information, reflecting the care and attention patterns. Hence, identifying and establishing indicators is a crucial step to define the important and representative aspects and values in the intervention. Based on such understanding, the assessment and reference devices can be developed for the whole program.

Measuring and monitoring the indicators have many purposes - e.g., estimating the quality of the health care or intervention, making judgments, establishing priorities, supporting administrators and leaders, and improving the quality of the service^11,12.

Validated indicators are expected to give the basis on which to construct assessment instruments that meet and represent quality, control, and follow-up functions of the components involved in the HCP. These are useful for the program and company administrators to monitor and assess the effectiveness and efficiency of the program.

The benefits of the assessment approaches make it possible to identify trends and improve the program, reflecting on the quality of the service, the efforts, and the expected results. Given the above, this research aimed to validate indicators and parameters to construct an HCP assessment instrument.

Methods

The study was approved by the Research Ethics Committee of the Universidade Federal de Pernambuco (Federal University of Pernambuco), Brazil, under evaluation report number 1.978.729. It was developed in four stages, as described below.

This is methodological research with a quantitative and qualitative approach, developed from June 2018 to May 2019.

The sample, comprising 20 speech-language-hearing therapists, was selected by convenience with the snowball technique. The key informants (n = 3) were selected based on their curriculum, which was consulted in the Lattes platform of the National Council for Scientific and Technological Development (CNPq, its Portuguese abbreviation). They were first contacted via e-mail, as registered in the platform, whereas the other participants were initially contacted via phone call or e-mail, as indicated by the key informants.

Stage 1: Establishing criteria, indicators, and parameters of the HCP

In the first stage of the research, the researchers surveyed the assessment indicators and parameters of the program, based on a Logical Operating Model of the HCP¹³. Also, the Manual of Guidelines and Parameters to implement the HCP was consulted¹⁴ as a complementary document to survey the indicators. The surveyed indicators and parameters made up an analysis and judgment matrix (AJM), whose items were submitted for validation.

Stage 2: Content and layout validation of the indicators and parameters to assess the HCP

The content validation consists in evaluating whether a given concept represents the extent of the phenomenon of interest and whether each item encompasses that which it is meant to investigate^15,16. The layout validation judges the relevance and adequacy of the form in which the items are presented¹⁷.

Ten speech-language-hearing therapists included in this stage comprised the panel of experts. Most of the participants were females (n = 9; 90%), mean age 42.6 years (SD = 10.95), with a specialization (90%) and a doctor’s degree (10%), who worked in the private sector (90%) administering HCP (80%). Their professional experience with the program ranged from 3 to 25 years (mean = 11.1; SD = 8.03).

The AJM of the HCP allowed for the development of partially open-ended questionnaires to validate the content and layout of the items, structured in Google Forms^® (^{Appendix A}). In each collection round, the questionnaire was accessed with a link to the electronic form, sent via e-mail to the participants.

The content and layout of the items were validated with the Delphi method, which consists of a series of rounds where a group makes judgments and comments on criteria and parameters of a given phenomenon to reach a consensus. The method requires confidentiality, the participation of experts, interactive rounds with feedback, and an effort to reach a consensus¹⁸.

At first, each item proposed in the AJM was assessed based on different criteria, with an adapted ordinal four-point Likert-type scale that categorized the items as (1) inadequate, (2) little adequate, (3) quite adequate, and (4) fully adequate. Using an ordinal four-point scale is recommended to avoid a neutral, ambivalent middle score¹⁹. The other phases in the Delphi method included administering questionnaires with clearer and more detailed formulations to make the study more understandable to those involved. Then, the comments, critiques, and opinions regarding the content and layout of the items were considered^18,20. A schematic drawing of the application steps of the Delphi method is shown in Figure 1.

Figure 1:
Representation of the consensus rounds based on the Delphi method.

The criteria for content validity of the items considered their appropriateness, objectivity, simplicity, clarity, and precision. As for layout validity, the criteria encompassed presentation, clarity of the statements, readability, interpretation, and representativity. Also, some room was made available for the experts to make observations in the assessment of the items.

Concerning the verification parameters of the indicators proposed in the AJM, the suggested scores for each indicator ranged from one (1.0) to three (3.0) points. They were then evaluated according to their appropriateness.

The data from the validation process were tabulated in Microsoft Office Excel 2010 and treated quantitatively based on descriptive statistical analysis. The qualitative analyses were conducted with content analysis²¹, whose stages comprised a pre-analysis of the experts’ observations and suggestions, followed by an exploration of the material and treatment of the results, establishing and organizing the recurrent theme categories.

The results of the content validation process, specifically the assessment of the appropriateness of the items and their scores, were analyzed based on the item content validity index (I-CVI) and scale content validity index (S-CVI). The condition established to validate each item was I-CVI and S-CVI values equal or superior to 0.80, categorizing them as adequate²². The items that scored “1” or “2” were revised in the subsequent phases of the Delphi method and submitted to new judgment by the judges until a consensus was reached.

The content and layout validity criteria were analyzed with the percentage of absolute agreement, obtained by summing the items that were assessed positively and dividing it by the total of answers. The criteria that achieved 80% or more were considered valid.

Stage 3: Construction of the HCP assessment instrument

The validated items of the AJM of the HCP were the basis for the construction of the first version of the program assessment instrument as a structured questionnaire. The normative assessment instrument makes it possible to judge the assessment items in contrast with the expected verification parameters (norm/score standard), whose result enables the program’s degree of implementation to be estimated with different cutoff scores.

The manner how the programs are implemented may be as important as the results, especially when they involve the participation of various professionals and their integration with the administrators in the implementation process²³. Considering the importance and complexity of the processes implemented in the HCP, these components’ score was multiplied by six, whereas the structure components’ score was multiplied by four for the analysis. Thus, the total degree of implementation was established by calculating the weighted mean of the answers obtained in the application of the instrument, with the formula presented in Figure 2, whose result was presented in percentages. The value judgment of the degree of implementation considered four cutoff scores (< 25.0% - not implemented; from 25.1% to 50.0% - incipient implementation; from 50.1% to 75.0% - partially implemented; and > 75.1% - implemented).

$\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{}\mathrm{D}\mathrm{I}=\left(\frac{\left(4\sum S^1+6\sum P^1\right)∕10}{\left(4\sum S^2+6\sum P^2\right)∕10}\right)\mathrm{*}100$

Figure 2:
Formula to calculate the degree of implementation of the hearing conservation program.

Stage 4: Semantic validation of the HCP assessment instrument

The semantic validation consists in the theoretical analysis of the items proposed in the instrument, approaching the understanding and acceptance of the terms, the relevance of the items, the existence of difficulties, and the possible need for adjustments¹².

Ten speech-language-hearing therapists participated in this stage, most of whom were females (n = 9; 90%), mean age 32.7 (SD = 7.0), with specializations (70%) and master’s degree (20%). Most of these professionals worked in the private sector (70%), with professional experience ranging from 2 to 9 years (mean = 4.5; SD = 2.61).

The data from the semantic validation of the HCP assessment instrument were collected by administering the initial version of the instrument itself to the participants, as well as two validation questionnaires based on the research of the European group DISABKIDS²⁴. In this approach, a general impression questionnaire was proposed (general semantic validation) along with a specific semantic validation questionnaire for the HCP assessment instrument.

The analyses were conducted with the percentage of absolute agreement, considering valid the criteria with values equal to or higher than 80%. The experts’ observations and suggestions underwent a content analysis²¹, whose data were analyzed and categorized, justifying the necessary readjustments in the instrument.

Results

Considering the HCP Logical Operating Model, 64 indicators were defined, 28 items referring to the structure component, and 36, to the processes, which were the basis to create the first version of the AJM of the HCP.

The AJM was developed considering the HCP dimensions, criteria, indicators, and standard (norms) expected for each indicator and the respective verification parameters (expected score and cutoffs). The criteria represent one or more aspects of the action or service and are represented by indicators, which work as attributes used to measure the components of a program²⁵. The norms or standard is the conceivable value of the criterion.

The response rates obtained in the content and layout validation process were 47.6% (10/21) in the first round of the Delphi method (Stage 2), 90% (9/10) in the second round, and 66.6% (6/9) in the third and last round. Everyone’s answers were considered for analysis, even of those who had stopped participating in previous rounds - hence, 10 participants were considered for this stage.

A difficulty commonly found in using this methodology is that some participants abandon it from one round to the other, either because the process takes too long or because it requires too much time²⁶. However, since the results are analyzed by the researchers in between each round of questionnaires, it is important that all the opinions - including the minority ones of participants who left it - be also shared and reported. Thus, the panel of experts has access to them and can include them in their considerations and arguments²⁷.

Two structure indicators out of the items proposed in the first validation round were not considered appropriate - i.e., these indicators were not considered relevant, as they did not meet the intended purpose. Item 20, which refers to the “Existence of an Awareness Statement”, was considered appropriate by only 60% (I-CVI = 0.60) of the experts. Item 22, which refers to the “Existence of a duplicate of the Occupational Health Certificate (OHC)” was considered appropriate by 70% (I-CVI = 0.70) of the experts.

Altogether, the items obtained a representative agreement level (S-CVI = 0.98), and their amplitude and balance were considered adequate. The score of all the items was considered appropriate (S-CVI = 0.93).

Some items (n = 6), whose appropriateness remained valid, did not reach a considerable agreement in one or more content assessment criteria (Table 1), while others (n = 14) did not reach a considerable agreement in one or more layout assessment criteria (Table 2). In other cases, despite having an adequate content, layout, and scores, the items (n = 12) received comments and/or suggestions that were included for the second consensus round, leading to revisions in its presentation.

Table 1:
Items that did not achieve an acceptable percentage of absolute agreement in relation to the content validity criteria regarding the structure and process indicators of the hearing conservation program

Indicators	Percentage of Absolute Agreement
	Objectivity	Simplicity	Clarity	Precision
2. Existence of Specialized Safety Engineering and Occupational Medicine Services (SESMT), or a person or team that, according to the employer, can develop engineering projects and measures.	90%	60%	60%	80%
3. Existence of Specialized Safety Engineering and Occupational Medicine Services (SESMT), or a person or team that, according to the employer, can develop the occupational health medical control program.	100%	60%	60%	80%
9. Existence of a risk analysis report	60%	80%	80%	90%
10. Existence of an Environmental Risk Prevention Program (ERPP) planning report	80%	80%	90%	70%
20. Existence of an Awareness Statement	80%	80%	70%	90%
26. Existence of educative material	80%	60%	60%	80%

Table 2:
Items that did not achieve an acceptable percentage of absolute agreement in relation to the layout validity criteria regarding the structure and process indicators of the hearing conservation program

Indicators	Percentage of Absolute Agreement
	Form of presentation	Readability	Clarity in the presentation	Interpretation	Representativity
2. Existence of Specialized Safety Engineering and Occupational Medicine Services (SESMT), or a person or team that, according to the employer, can develop engineering projects and measures.	60%	70%	70%	70%	70%
3. Existence of Specialized Safety Engineering and Occupational Medicine Services (SESMT), or a person or team that, according to the employer, can develop the occupational health medical control program.	60%	60%	80%	70%	70%
4. Existence of Specialized Safety Engineering and Occupational Medicine Services (SESMT), or a person or team that, according to the employer, can develop the hearing conservation program.	50%	80%	70%	80%	70%
10. Existence of an Environmental Risk Prevention Program (ERPP) planning report.	80%	80%	80%	70%	80%
12. Existence of a hearing protection device (HPD) attenuation review.	90%	70%	90%	100%	80%
17. Existence of audiometric examination forms	70%	100%	90%	100%	100%
20. Existence of an Awareness Statement	70%	80%	70%	80%	80%
21. Existence of an individual clinical record	70%	90%	90%	90%	90%
22. Existence of a duplicate of the occupational health certificate (OHC)	70%	70%	60%	80%	70%
26. Existence of educative material	90%	90%	70%	90%	80%
27. Existence of informational material	80%	90%	70%	80%	80%
33. Definition of responsibilities, goals, and priorities of the hearing conservation program (HCP)	90%	90%	70%	100%	90%
36. Participation in the implementation of engineering measures	90%	90%	70%	100%	90%
37. Participation in the implementation of administrative measures	100%	90%	70%	80%	90%

In the second consensus round, most of the items proposed (n = 23) were revised - and then their content and layout reached a consensus - aiming at greater objectivity, simplicity, clarity, and precision of the statements, as well as improvements in their presentation, readability, clear presentation, interpretation, and representativity. In this round, new suggestions came up regarding some items (n = 9), which were presented for consensus in the third round.

After the third and last validation round, all the suggestions reached a consensus. Altogether, 62 items were validated, considering 26 indicators related to structure and 36, to processes. A representation of the rounds is given in ^{Annex 1}.

The AJM provided valid means to develop the first version of the HCP assessment instrument to estimate the degree of implementation of the program. It is a structured, standardized questionnaire, whose structure includes information on the company, the assessment, presentation of the instrument with instructions to fill it in, two sets of questions regarding the items to be assessed (resources and activities), a sheet for interpreting and analyzing the results to establish the degree of implementation of the HCP, and in the end a sheet to present the results and indicate necessary improvements.

There were 26 assessment questions in the first set, approaching the resources necessary for an efficient implementation of the HCP. These work as the basis for the actions to be implemented. The second set comprised 36 questions regarding the processes (actions) to be implemented, that is, the activities that need to be carried out in the program.

Each question has a description of the expected standard answer, setting a parameter for the assessment. Considering the appropriateness validity of the scores proposed, it was established that the expected summed score of the set of structure items of the HCP would be 55.0 points, while that of the process items would be 87.0 points; hence, the overall expected score was established at 142 points. The existence of resources and the implementation of processes according to the established parameters must be given the total expected score. The absence or non-presentation of resources, as well as the non-implementation or the partial implementation of processes, are not given any score.

The scores obtained in each set are analyzed according to the instructions given in the sheet for the interpretation and analysis of results, present in the instrument. The analysis of the results must consider the scores obtained in relation to the one expected in each set of the assessment. The total degree of implementation is then presented in percentage, resulting from the calculation of the weighted mean of the results obtained in relation to the expected in each set. The degree of implementation is classified based on the cutoff scores distributed into four tiers²⁵, as described below:

1. Not implemented: < 25%

2. Incipient implementation: 25.1% to 50%

3. Partially implemented: 50.1% to 75.0%

4. Implemented: >75.1%

In the semantic validation of the HCP assessment instrument (Stage 4), all the participants (n = 10) considered the instrument good (20%) or very good (80%). The participants assessed the questions present in the instrument mostly as “easy to understand” (90%), while there were “some difficulty” (10%). The answer options were assessed as “easy to understand” (90%) or “with some difficulties” (10%). Everyone considered the questions present in the instrument very important to assess the HCP. Most of them had nothing to change or add to the questionnaire (90%), and they felt comfortable answering all the questions included in the assessment instrument.

Some suggestions were made, leading the instrument to be revised. The analysis categories in consideration were in the field of information on the company, instructions to fill in the instrument, and the sheet of interpretation and analysis of the results. The final degree of implementation of the HCP is shown in Annexe 1.

Discussion

Considering the results obtained, it can be pointed out that, in general, the HCP process items easily reached a consensus among the experts. Only two of the items regarding this component were not considered valid and the suggestions and observations made while the other items reached a consensus dealt with the form of presentation of the indicator, suggesting clearer and more objective statements.

There are, in the literature, many suggestions on the minimally necessary stages to effectively implement the HCP activities. Among them, there is the practical guide proposed by NIOSH²⁸, a pioneer and reference agency in the development of research and guidelines involving the HCP and, more recently in the national context, the Manual of Guidelines and Minimal Parameters to Develop and Administer the Hearing Conservation Program, issued by FUNDACENTRO¹⁴. Based on these, the actions developed in the HCP are admitted and implemented as a consensus among administrators. However, the necessary structures for an efficient implementation of the program - considering the physical, human, and organizational resources - are not clearly defined in the guides. In this context, the items proposed regarding the HCP structure components raised many considerations by the experts, and it was necessary to reformulate them.

The content, layout, and form of presentation of the item “Existence of an Awareness Statement” were not considered clear (percentage of absolute agreement = 70%), neither was it considered appropriate (I-CVI = 0.60). The hearing loss awareness statement is a document in which the employee, after reading and signing it, states that they have had auditory rest for 14 hours and then undergone audiometric examination and that they have been informed of the result in case of hearing loss.

Such a practice is not provided by law. Nevertheless, it has been implemented in companies to have it registered as a means of protection because of the employees’ mandatory 14-hour auditory rest before undergoing audiometric examinations and the companies’ obligation to inform the result to the employees, especially when there is a hearing loss.

The law provides the auditory rest before undergoing initial and/or reference audiometric examinations^2,29. However, it is unclear whether the auditory rest is mandatory before sequential examinations unless its result is suggestive of triggering NIHL. In this case, the audiometry is defined as either reference or sequential, thus requiring a new evaluation in light of the guidelines proposed by the Regulatory Norm no. 7, considering the auditory rest²⁹.

The time of auditory rest - which is recommended in Service Order 608 - is necessary for the auditory effects caused by the high sound pressure levels (such as a temporary threshold shift) not to interfere with the result of the exam². In practical terms, considering the information in the audiological anamnesis, in case the employee has been exposed to high sound pressure levels a little before undergoing the examination and there are changes in the auditory thresholds in relation to the initial/reference examination, they are asked to return 24 hours later for a new examination.

The experts understand that “it is up to the company to ensure that the employee has time for the auditory rest”. The Regulatory Norm no. 7 establishes that “the employee will have an auditory rest for at least 14 hours before undergoing the examination”²⁹ - which is one of the basic principles and procedures to perform an audiometric examination. On the other hand, Service Order 608 does not explicitly point out this responsibility. Hence, the law does not clearly indicate whose responsibility it is to ensure the auditory rest, neither is there instruction on how to effectively control such time.

Also, the experts’ question “the feasibility of a 14-hour auditory rest for sequential examination because it implies in releasing the employee from their work duties”.

This deadlock leads companies to use strategies to meet the guidelines and better administer the diagnoses. These include the implementation of educative³⁰ and organizational measures, scheduling the work in accordance with the planned examinations. Also, the employees can be informed and motivated to use the necessary preventive and auditory rest strategies before undergoing an audiometric examination.

Another aspect brought up by the experts was that “the employee signs the result of the examination”, thus stating their awareness. The Regulatory Norm no. 7 recommends that reference auditory examinations be established, especially when hearing losses are present or triggered, comparing them with sequential ones for monitoring purposes²⁹. Likewise, Service Order 608 instructs that the employees must be informed of the results of each examination they are submitted to and receive a copy of them - although the diagnosis of a possible hearing loss does not disqualify the employee to perform their work duties². Nonetheless, the employee’s signature on the audiological examination does not ensure that such information was actually passed on to them - this practice only registers and proves that the examination was made.

Also, the experts did not consider the item that refers to the “Existence of a duplicate of the Occupational Health Certificate (OHC)” as an HCP indicator (I-CVI = 0.70). The OHC, a reference document of the Occupational Health Medical Control Program (OHMCP), is a certificate issued by the occupational physician. Besides stating they are apt to work, the certificate points out the specific occupational hazards and the medical procedures used, including complementary examinations²⁹. The experts argued that “the OHC indicates whether the employee is apt to perform their duties in the company” and that it “evaluates whether the employee’s health is in accordance with the risk exposure in the activities they must perform” and helps “the occupational safety administration”. Hence, it is not relevant to the HCP.

The law points out that having a hearing loss induced by high sound pressure levels does not in and of itself indicate the person is unfit for work. Each case must be analyzed separately, considering the auditory requirements of the work or position²⁹. Hence, the one responsible for defining whether the employee is fit for work when they are with suspicion for hearing loss induced by high sound pressure levels is each company’s OHMCP coordinating physician. They must consider the assessment of the type and degree of hearing loss presented by the employee, the duties they are to perform, the frequent audiological control, and educative actions³¹.

It should be highlighted that the OHC must state the risks the employee will be exposed to while performing their work activities. This is one of the indicators for occupational safety administration, revealing the need for audiological administration and monitoring of employees exposed to high sound pressure levels²⁹.

The definition of the structure indicators related to human resources regarding each one’s responsibilities to perform different actions led the experts to put many propositions forth. The human resources include the people involved in implementing the intervention - an essential aspect for the services to be adequately carried out.

The HCP, which is an interdisciplinary program with cooperation between sectors, is meant to relate different processes administered by different people - which poses a challenge for the program’s administration. Considering the processes involved in the implementation of the program, there supposedly is a need for the company’s professionals in the fields of health, safety, management, human resources (the company’s owner or personnel administrator), and, most of all, the employees who will implement the HCP^2,14. Including other professionals will depend on the complexity of the program, considering the company’s structure and context.

Concerning the “existence of Specialized Safety Engineering and Occupational Medicine Services (SESMT, its Portuguese abbreviation), person or team capable of developing engineering projects and measures” and the “participation in the implementation in the engineering measures”, there were different arguments about what professional would be able to implement such services and whether they should be developed by only one specialized professional or by a team, considering the occupational physician and the speech-language-hearing therapist. As for the law, Regulatory Norm no. 9 advocates that the actions of the Environmental Risk Prevention Program (ERPP) must be carried out by the SESMT, or by a person or team that, according to the employer, can develop what is provided in this Regulatory Norm³². Although it does not clearly specify what professionals have the responsibility, it can be said that having a specialized occupational safety engineer in the team that implements the HCP is utterly important. After all, this is the technically qualified professional to develop engineering measure projects. Including other necessary professionals and qualifications will depend on the comprehensiveness of the preventive measures proposed.

Also, regarding the person or team that can develop the HCP, there was no consensus at first concerning what professionals can implement and administrate the program, as such responsibilities are not clear in the law. The experts suggested to include, besides the speech-language-hearing therapist and occupational physician, the occupational safety engineer as one of the professionals who can administrate the HCP.

Considering the multidisciplinarity of the HCP, even before defining its administrator, it is important that the responsibilities be well-defined and delegated to technically capable professionals, considering the specificities of each stage to be implemented, its needs, and necessary competencies¹⁴. In this regard, a consensus was reached in terms of considering the existence of a technically capable professional or team to develop the program.

Along this line, there were different standpoints on the professional specialty responsible for carrying out the audiometric examinations. The law provides that they must be conducted only by a qualified professional (i.e., a speech-language-hearing therapist or a physician), as stated in resolutions of their professional councils, whereas no other specialty is pointed out^2,29. Hence, some experts argued that “the audiometry can be conducted by any medical professional, as long as they are specialized in audiology”. Others counterargued that, despite the regulatory law mentioning the physician, “few professionals in the field have the experience and skill to conduct the audiometric examinations; so, the speech-language-hearing therapist is the one who should do it”. After the following consensus rounds, the “existence of a professional specialized in audiology to conduct the audiological examinations” was established as a parameter for this indicator.

Concerning the “Inspection of personal protective equipment (PPE) use”, one of the experts suggested that this role “could be the responsibility of the occupational safety technician”. The suggestion was taken to the other experts for consensus, and they agreed that it is important to evaluate the HCP administrator’s cooperation with the person or team responsible for it and the other people of the program involved in inspecting the employees’ PPE use.

The employees’ PPE use is a complex habit, influenced by various factors, including intrapersonal (such as discomfort or interference with the communication), interpersonal (including the relationship with workmates, support personnel, and supervisors), and organizational factors (involving organization rules, spreading knowledge and information, and so on)³³. It is highly important to observe these factors associated with the aspects of health education and commitment of all those involved in the HCP in order to develop safe behaviors, particularly regarding PPE use.

To establish assessment parameters, aspects considered important and priority were surveyed, as they would reflect the quality of what is being assessed. In this sense, some structure indicators - as in the case of reports - had a set of variables to be observed in the assessment, specifically the resources related to the “Existence of a risk analysis report”, “Existence of an ERPP planning report”, “Existence of an OHMCP report”, “Existence of a PPE attenuation review”, “Existence of an HCP implementation report”, and “Existence of an individual clinical record”. Such items were proposed for a consensus in the following rounds, aiming at a better definition and clarity of the parameters to be assessed. It was generally agreed that it is important to specify the variables that would reflect actually important information for the HCP. Thus, the parameters were reformulated according to the experts’ suggestion and agreement.

Overall, the structure of the instrument was well-assessed by everyone regarding the fields of information on the company and on the assessment, the instructions to fill in the instrument, the sets of structure and process assessment, the sheet for the analysis and interpretation of answers, and the sheet of results, pointing out the clear description and understandability.

The semantic validity of the HCP assessment instrument reflected a satisfactory evaluation on the part of the consulted professionals. The form of presentation and listing of resources and processes to be assessed were deemed “well-written and easy to understand”, as pointed out by one of the professionals. They considered that “the listing and description of the necessary resources to implement the HCP are great”, and that “the sequence favors the identification of the actions to be implemented”.

The description of the expected parameters for each item was considered positive, as it “makes the assessor’s job easier, making it clearer what they have to observe”. The checklists of the actions to be implemented and assessed in the HCP are widely used since the publication by NIOSH²⁸, evidencing the importance of checking separately the quality and the full carrying out of each implementation stage of the program in the periodic audits. Research also points out that using a predeveloped assessment guide makes it easier to verify the functioning of the HCP and the detection of flaws in the program³⁴.

The professionals commented on the objective and functionality of the instrument, pointing out that the material “will enable the researcher to know the companies that have an effective HCP and the ones that still need to implement it”. The degree of implementation estimate is an alternative to assess the program implementation process. It gives the basis for administrators, assessors, and researchers to both solve possible problems in the operationalization of the intervention and to formulate recommendations aimed at improving the program. Such an estimate can be applied before, during, or after implementing the HCP³⁵.

Also, the participants highlighted the importance of the assessment instrument to monitor the program, stating that “it is good for the companies as well, for them to see what they are doing wrong”. The HCP assessment by audits aims to determine, among other things, whether the actions implemented in the program were effective to meet the objectives outlined for occupational health and safety, and whether the company complies with the existing legal requirements³⁶, thus revealing possible mistakes and deviations from the norm. However, other evaluative surveys must complement the results obtained from the audits³⁷. This makes it possible to understand the current condition of the program in relation to different aspects involved in the intervention - which may both influence the degree of implementation and be influenced by it - such as the contextual aspects (political and structural), the expected results, and their impacts. The degree of implementation then becomes an important variable to observe and analyze these conditions.

The validation of this instrument is necessary to obtain reliable measures (based on the application of the instrument in different contexts) and statistical analysis of stability, internal consistency, and equivalence. In this perspective, other psychometric properties of the instrument can be tested to verify the precision of the data and measures, which must be valid and interpretable to assess the HCP.

Conclusion

The validity of assessment indicators and parameters for the HCP proves to be important in the planning, analysis, and monitoring of the program. Hence, they provide the means to construct the evaluative instruments that encompass the operational components of the intervention, namely, the structure, processes, and expected results.

The instrument to estimate the degree of implementation of the HCP aims to reveal the current condition of the intervention, considering criteria, indicators, and parameters validated by experts. It can be self-applied or directed to the administrator of the program before, during, or after its implementation.

Acknowledgments

Gratitude is extended to the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) and to the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) for the financial support granted.

REFERENCES

• ¹
  Gomez CM, Vasconcellos LCF, Machado JMH. A brief history of worker's health in Brazil's Unified Health System: progress and challenges. Ciênc. saúde colet. [journal on the Internet]. 2018 [cited 2019 Mar 16]; 23 (6):1963-70. Available at: https://www.scielosp.org/article/csc/2018.v23n6/1963-1970/
  » https://www.scielosp.org/article/csc/2018.v23n6/1963-1970/
• ²
  Brasil. Ministério da Previdência e Assistência Social. OS/INSS nº608, de 05/08/1998. Norma Técnica sobre Perda Auditiva Neurossensorial por Exposição Continuada a Níveis Elevados de Pressão Sonora de Origem Ocupacional. Brasília: Diário Oficial da República Federativa do Brasil; 1998.
• ³
  Brueck SE, Prince PM, Stancescu D, Woskie S, Estill C, Waters M. Noise exposure reconstruction and evaluation of exposure trends in two large automotive plants. Ann Occup Hyg. 2013;57(9):1091-104.
• ⁴
  Mooyad MA. The Effectiveness of Hearing Conservation Program in the workplace. Glob J Oto. 2017;5(4):555666
• ⁵
  Oliveira WTGH, Andrade WTL, Teixeira CF, Lima MLLT. Audição de trabalhadores antes e após o Programa de Conservação Auditiva. Rev. bras. ciênc. Saúde. 2013;16(4):517-24.
• ⁶
  Sayler SK, Rabinowitz PM, Cantley LF, Galusha D, Neitzel RL. Costs and Effectiveness of Hearing Conservation Programs at 14 US Metal Manufacturing Facilities. Int J Audiol. 2017;57(1):3-11.
• ⁷
  Faria HP, Campos FCC, Santos MA. Planejamento e avaliação das ações em saúde. Belo Horizonte: Nescon/UFMG, Coopmed; 2010.
• ⁸
  Feitoza ANC, Oliveira CL, Duarte EB, Oliveira AM. Health evaluation: a integrative review. Id on Line Rev. Psic. [journal on the Internet]. 2016 [cited 2019 Feb 12]; 10 (30):274-81. Available at: https://idonline.emnuvens.com.br/id/article/view/457
  » https://idonline.emnuvens.com.br/id/article/view/457
• ⁹
  Antero AS. Monitoramento e avaliação do Programa de Erradicação do Trabalho Escravo. Rev. de Adm. Publica. 2008;42(5):791-828.
• ¹⁰
  Cruz MM. Usos do planejamento e autoavaliação nos processos de trabalho das equipes de Saúde da Família na Atenção Básica. Saúde debate. 2014;38(spe):124-39.
• ¹¹
  Báo ACP, Amestoy SC, Moura GMSS, Trindade LL. Quality indicators: tools for the management of best practices in Health. Rev. bras. enferm. [journal on the Internet]. 2019 [cited 2019 Dez 27]; 72(2):360-6. Available at: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034 71672019000200360&lng=en
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034 71672019000200360&lng=en
• ¹²
  Occupational Safety and Health Administration (OSHA). Using Leading Indicators to Improve Safety and Health Outcomes. U.S.: Department of Labor; 2019 [cited 2020 Jan 5]. Available at: https://www.osha.gov/leadingindicators/docs/OSHA_Leading_Indicators.pdf
  » https://www.osha.gov/leadingindicators/docs/OSHA_Leading_Indicators.pdf
• ¹³
  Pimenta AS, Teixeira CF, Silva VM, Almeida BGP de, Lima MLLT. Logical operating model of the hearing conservation program for workers. Rev. CEFAC [journal on the Internet]. 2019 [cited 2019 Oct 10]; 21(3):e16018. Available at: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1516
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1516
• ¹⁴
  Fundacentro. Guia de diretrizes e parâmetros mínimos para a elaboração e a gestão do PCA; 2018 [cited 2019 Jan 20]. Available at: http://www.fundacentro.gov.br/biblioteca/biblioteca-digital/pu blicacao/detalhe/2018/9/guia-de-diretrizes-e-parametros-minimos-para-a-elaboracao-e-a-gestao-do-pca
  » http://www.fundacentro.gov.br/biblioteca/biblioteca-digital/pu blicacao/detalhe/2018/9/guia-de-diretrizes-e-parametros-minimos-para-a-elaboracao-e-a-gestao-do-pca
• ¹⁵
  Medeiros RKS, Ferreira JMA, Pinto DPSR, Vitor AF, Santos VEP, Barichello E. Modelo de validação de conteúdo de Pasquali nas pesquisas em Enfermagem. Rev. Enf. Ref. [journal on the Internet]. 2015 [cited 2019 Jul 12]; ser IV(4):127-35. Available at: http://www.scielo.mec.pt/scielo.php?script=sci_arttext&pid=S0874-02832015000100014&lng=pt
  » http://www.scielo.mec.pt/scielo.php?script=sci_arttext&pid=S0874-02832015000100014&lng=pt
• ¹⁶
  Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar AR. Design and Implementation Content Validity Study: Development of an instrument for measuring Patient-Centered Communication. J Caring Sci. [journal on the Internet]. 2015 [cited 2019 Jul 12]; 4(2):165-78. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4484991/pdf/jcs-4-165.pdf
  » https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4484991/pdf/jcs-4-165.pdf
• ¹⁷
  Souza AC, Alexandre NMC, Guirardello EB. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol. Serv. Saúde [journal on the Internet]. 2017 [cited 2019 Jul 12]; 26(3):649-59. Available at: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S223796222017000300649&lng=en
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S223796222017000300649&lng=en
• ¹⁸
  Marques JBV, Freitas D. Método DELPHI: caracterização e potencialidades na pesquisa em Educação. Pro-Posições [journal on the Internet] 2018 [cited 2018 Dec 15]; 29(2):389-415. Available at: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0103-73072018000200389
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0103-73072018000200389
• ¹⁹
  Lynn MR. Determination and quantification of content validity. Nursing Research. 1986;35(6):382-5
• ²⁰
  Massaroli A, Martini JG, Lino MM, Spenassato D, Massaroli R. The Delphi Method as a methodological framework for research in nursing. Texto contexto - enferm. [journal on the Internet]. 2017 [cited 2018 Jun 15]; 26(4):e1110017. Available at: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072017000400320&lng=en
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-07072017000400320&lng=en
• ²¹
  Bardin L. Análise de conteúdo. São Paulo: Edições 70; 2011.
• ²²
  Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Social Work Research. 2003;27(2):94-104.
• ²³
  Furtado JP, Campos GWS, Oda WY, Onocko-Campos R. Planejamento e Avaliação em Saúde: entre antagonismo e colaboração. Cad. Saúde Pública [journal on the Internet]. 2018 [cited 2019 Dec 26], 34(7):e00087917. Available at: https://doi.org/10.1590/0102-311X00087917
  » https://doi.org/10.1590/0102-311X00087917
• ²⁴
  Guedes DM, Rossato LM, Ramos MC, Borghi CA, Carvalho JA. Pain assessment in children: semantic adequacy of a multidimensional instrument in a context of the northeast region. Rev Soc Bras Enferm Ped. [journal on the Internet] 2018 [cited 2019 Nov 12]; 18(2):82-9. Available at: https://sobep.org.br/revista/images/stories/pdf-revista/vol18-n2/sobep-18-2-artigo-original-04.pdf
  » https://sobep.org.br/revista/images/stories/pdf-revista/vol18-n2/sobep-18-2-artigo-original-04.pdf
• ²⁵
  Samico I, Felisberto E, Figueiró AC, Frias PG. Avaliação em saúde: bases conceituais e operacionais. Rio de Janeiro: MedBook; 2010.
• ²⁶
  Sáfadi CMQ. Delphi: um estudo sobre sua aceitação. In: V SEMEAD [internet]; 2001; São Paulo, SP; 2001 [cited 2018 Oct 28]. Available at: http://sistema.semead.com.br/5semead/MKT/Delphi.pdf
  » http://sistema.semead.com.br/5semead/MKT/Delphi.pdf
• ²⁷
  Facione PA. Critical thinking: a statement of expert consensus for purposes of educational assessment and instruction. Research findings and recommendations (Report). Newark: American Philosophical Association; 1990.
• ²⁸
  National Institute for Occupational Safety and Health (NIOSH). Preventing occupational hearing loss - a practical guide. Department of Health and Human Services - Centers for Disease Control and Prevention; 1996.
• ²⁹
  Brasil. Ministério do Trabalho e Emprego. Portaria MTb nº 3.214, de 08 de junho de 1978. Norma Regulamentadora nº7 - Programa de Controle Médico de Saúde Ocupacional [Internet]. Brasília: Diário Oficial da República Federativa do Brasil, 06/07/78. [cited 2016 Ago 10]. Available at: http://trabalho.gov.br/images/Documentos/SST/NR/NR7.pdf
  » http://trabalho.gov.br/images/Documentos/SST/NR/NR7.pdf
• ³⁰
  Bramatti L, Morata TC, Marques JM. Ações educativas com enfoque positivo em programa de conservação auditiva e sua avaliação. Rev. CEFAC. 2008;10(3):398-408.
• ³¹
  Gatto CI, Lermen RA, Teixeira TM, Magni C, Morata TC. Análise da consulta de médicos de trabalho diante de trabalhadores com perda auditiva. Distúrb. Comunic. 2005;17(1):101-13.
• ³²
  Brasil. Ministério do Trabalho e Emprego. Portaria GM/SSSTb nº25, de 29/12/1994. Norma Regulamentadora N°9 - Programa de Prevenção de Riscos Ambientais [Internet]. Brasília: Diário Oficial da República Federativa do Brasil, 30/12/1994. [Accessed on 2016 Ago 10]. Available at: http://trabalho.gov.br/images/Documentos/SST/NR/NR-09.pdf
  » http://trabalho.gov.br/images/Documentos/SST/NR/NR-09.pdf
• ³³
  Tinoco HC, Lima GBA, Anna APS, Gomes CFS, Santos JAN. Risk perception in the use of personal protective equipment against noise-induced hearing loss. Gestão & Produção. 2019;26(1): e1611
• ³⁴
  Cavalli RCM, Morata TC, Marques JM. Auditoria dos programas de prevenção de perdas auditivas em Curitiba (PPPA). Rev. Bras. Otorrinolaringol. [journal on the Internet]. 2004 [cited 2018 Jan 17]; 70(3):368-77. Available at: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-72992004000300013&lng=en http://dx.doi.org/10.1590/S0034-72992004000300013.
  » https://doi.org/http://dx.doi.org/10.1590/S0034-72992004000300013» http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0034-72992004000300013&lng=en
• ³⁵
  Vieira-Da-Silva LM. Conceitos, abordagens e estratégias para a avaliação em saúde. In: Hartz ZMA, Silva LMV (orgs). Avaliação em saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Rio de Janeiro: Fiocruz; 2005. p. 15-40.
• ³⁶
  Gonçalves GCO. Saúde do Trabalhador: da estruturação à avaliação de programas de preservação auditiva. 1th ed. São Paulo: Roca; 2009.
• ³⁷
  Bettega A. Avaliação da eficácia do programa de prevenção de perda auditiva em uma indústria brasileira [Dissertation]. Curitiba (Paraná): Universidade Tuiuti do Parana; 2010.

• Research support source: Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq); Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) - Finance Code 001.

APPENDIX A

Validation questionnaire regarding the first round applying the Delphi Method

ANNEX 1

Assessment instrument of the Hearing Conservation Program

INFORMATION ABOUT THE COMPANY

CNPJ: ________________ COMPANY NAME: ____________________________________________________________ TRADE NAME: _____________________________________________________________________________________ CONTACT (Name/Position/Phone): ______________________________________________________________________ TYPE: ( ) Manufacturing ( ) Commerce ( ) Services ( ) Other: ________________________________ AVERAGE NUMBER OF EMPLOYEES: ( ) Up to 10 ( ) 11 to 100 ( ) 101 to 500 ( ) More than 500 Is there an Environmental Risk Prevention Program (ERPP): ( ) Yes ( ) No Is there an Occupational Health Medical Control Program (OHMCP)? ( ) Yes ( ) No Is there a Hearing Conservation Program (HCP) at the company? ( ) Yes ( ) No In case there is a Hearing Conservation Program at the company, fill in the date when the service was first implemented: _____________________________________________________ In case there is a Hearing Conservation Program at the company, fill in the name and position of the administrator of the program: _________________________ /___________________________

INFORMATION ABOUT THE ASSESSMENT

DATE OF THE ASSESSMENT: _________________________________________________________________________ NAME OF THE ASSESSOR: ___________________________________________________________________________ CPF/CNPJ: _______________________________ CONTACT: ________________________________________________ TYPE OF ASSESSOR: ( ) EXTERNAL ( ) INTERNAL (Position in the company: ______________) REQUESTER: _______________________________________________________________________________________ CPF/CNPJ: __________________________________________________________________________________________ TYPE OF ASSESSMENT: ( ) First implementation of the Hearing Conservation Program ( ) Implementation monitoring of the Hearing Conservation Program with a new administration ( ) Implementation monitoring of the Hearing Conservation Program with the current administration ( ) Annual assessment of the Hearing Conservation Program ( ) Normative basis for evaluative research of the Hearing Conservation Program FORM OF ASSESSMENT ( ) Self-assessment ( ) Direct observational ( ) Indirect - Questionnaire filled in by the administrator of the program INTERVIEWEE: ____________________________________________________________________________________ POSITION IN THE COMPANY: _______________________________________________________________________

INSTRUCTION TO FILL IN THE INSTRUMENT

This assessment instrument was developed considering structure (resources) and process (activities) indicators that make up the Hearing Conservation Program, based on the existing national law and with a consultation to scientific evidence and documents. This questionnaire was constructed based on the following documents: Regulatory Norm No. 6 of the Ministry of Labor and Employment (NR 6); Regulatory Norm No. 7 of the Ministry of Labor and Employment (NR 7); Regulatory Norm No. 9 of the Ministry of Labor and Employment (NR 9); Regulatory Norm No. 15 of the Ministry of Labor and Employment (NR 15); Norm of Occupational Hygiene 1 (NHO-1) of FUNDACENTRO; Service Order 608 (OS 608) of the National Social Security Institute; Manual of Guidelines and Minimal Parameters to Develop and Administer the Hearing Conservation Program of FUNDACENTRO. The instrument aims to estimate the degree of implementation of the HCP - i.e., the condition/level in which the program was/is being implemented. The instrument is divided into two sets. In the first assessment set, there are 26 questions on the resources necessary to effectively implement the Hearing Conservation Program. These are the basis for the actions that will be implemented. The second set comprises 36 questions on the processes (actions) to be implemented - i.e., the activities that need to be developed in the program. Each question has a description of the expected answer, setting an assessment parameter. In the assessment of resources, the maximum expected score is 55.0 points. As for the processes, the maximum expected score is 87.0 points. The total expected score for the instrument is 142 points, which would reflect a fully implemented Hearing Conservation Program. The existence of resources and the implementation of processes following the parameters specified in the instrument must be given the total expected score for each question. The absence or non-presentation of the resources, as well as the non-implementation or partial implementation of processes, are not given any score. The information can be collected from the administrator of the Hearing Conservation Program or the administrator of the Occupational Health Medical Control Program, depending on the type and form of assessment. For the analysis of the answers, the scores obtained in each set are summed, and these results are applied in the formula to achieve the degree of implementation of the Hearing Conservation Program, as presented in the sheet for Interpretation and Analysis of the results. Lastly, the sheet with Results of the Assessment of the Degree of Implementation of the Hearing Conservation Program is filled in, in which the assessor presents the final result of the assessment, the aspects that need improvement, complementary observations, and further instructions aiming at the improvement of the program.

SET I - ASSESSMENT OF THE STRUCTURE OF THE HEARING CONSERVATION PROGRAM
PHYSICAL, HUMAN, AND ORGANIZATIONAL RESOURCES
EVALUATIVE QUESTION	EXPECTED STANDARD/NORM	EXPECTED SCORE	SCORE OBTAINED
Is there a person or position in the company responsible for ensuring the resources and administrative measures to carry out the Hearing Conservation Program?	Existence of (1) Specialized Safety Engineering and Occupational Medicine Services (SESMT) and/or (2) at least one department, person, or representative directly related to the company.	3.0
Is there in the company a Specialized Safety Engineering and Occupational Medicine Service (SESMT), or person or team that can develop engineering projects and measures?	Existence of an engineer specialized in occupational safety or an interdisciplinary team that includes this professional	2.0
Does the company have an occupational physician responsible for the occupational health medical control program?	Existence of an occupational physician	3.0
Does the company have a professional or team technically qualified to develop the Hearing Conservation Program?	Existence of an occupational physician and/or speech-language-hearing therapist and/or occupational safety engineer	3.0
Does the company have a risk analysis report?	Existence of an annual risk analysis report, presenting at least general data of assessment and description of the exposure risks and conditions.	3.0
Does the company have an Environmental Risk Prevention Program planning report?	Existence of an annual report or whenever there is any change in the risk analysis report, presenting the (1) definition of goals, priorities, and schedule; (2) action strategy and methodology (according to the hierarchy of the control measures); (3) form of the registry, maintenance, and communication of the data; (4) Periodicity and form of assessment of the development.	3.0
Does the company have a report on the occupational health medical control program?	Existence of an annual report on the occupational health medical control program, presenting data from the administration of audiological diagnoses, with statistics of the results of nosologic diagnoses and evolution diagnoses by departments of the company.	3.0
Does the company have an implementation report of the Hearing Conservation Program?	Existence of an annual report and/or when there is a new administration of the Hearing Conservation Program, presenting the (1) contextualization of the company and objectives to be reached; (2) company’s policy regarding the Hearing Conservation Program; (3) responsibilities and competences; (4) analysis of the risks of occupational hearing loss; (5) suggestions of collective and/or personal control measures; (6) data of the administration of diagnoses (statistics of the results of nosologic diagnoses), audiological monitoring (evolution diagnoses); and (7) assessment of the program, in case it has been implemented before.	3.0
Is there any proof that the audiometric examination is performed in an audiometric booth or an acoustically treated environment?	Existence of a review of the sound booth or acoustically treated environment to perform the audiometric examinations, in compliance with ISO 8253-1.	3.0
Does the company provide informational material in its facilities?	Existence of bulletin boards and/o risk maps in the facilities of the company, presenting the risks to hearing health, indicating areas with high sound pressure levels.	2.0
Does the company have an attenuation review of the personal protective equipment (PPE) used by the employees?	Existence of an annual attenuation review of the personal protective equipment used by the employees, presenting the techniques used, the equipment analyzed, and the results obtained.	3.0
Does the company have personal protective equipment adequate to the risk of each activity?	Existence of hearing protection device(s), which can be circumaural hearing protection, insert hearing protection, or semi-auricular hearing protection.	1.0
Does the company have a Certificate of Approval of the personal protective equipment used by the employees?	Existence of a certificate of approval of the PPE, issued by the national department responsible for occupational health and safety issues in the Ministry of Labor and Employment.	2.0
Does the company have proof of receipt of the hearing protection devices?	Existence of a registry form of delivery/maintenance of the personal protective equipment.	1.0
Are there forms in the company to communicate accidents at work?	Existence of forms to communicate accidents at work related to the data in the reports of the Hearing Conservation Program and the Occupational Health Medical Control Program, to control employees with occupational hearing loss	1.0
Are there individual clinical records of the employees?	Existence of individual clinical records, presenting at least the analysis of the results obtained in the audiometric examinations, the definition of the nosologic examination, and analysis of the evolution and definition of the evolution diagnosis of occupational hearing losses.	2.0
Is there a specialized professional to carry out the audiological examinations?	Existence of a professional specialized in audiology to carry out the audiological examinations.	2.0
Has the professional proved their qualification to carry out the audiological examinations?	Existence of a receipt from the pertaining professional council proving the qualification as a physician or speech-language-hearing therapist who carries out the audiometric examinations.	3.0
Is there any proof that the audiometric examination is carried out with duly calibrated measuring equipment?	Existence of updated measuring/calibration certificate(s) of the equipment used for the audiometric examinations.	3.0
Does the service have a clinical-occupational questionnaire/anamnesis?	Existence of clinical-occupational questionnaire, including (1) occupation and position in the company; (2) current and previous exposure to high sound pressure levels; (3) exposure to other risks related to occupational hearing loss; (4) use of hearing protection device; (5) use of ototoxic medication; (6) family history of hearing loss; (7) non-occupational exposure to high sound pressure levels; (8) auditory and extra-auditory symptoms.	1.0
Does the service have audiometric examination forms?	Existence of audiometric examination forms complying with Table II, Annex I of NR7.	1.0
Are there proofs that a duplicate of the audiometric examination has been delivered to the employee?	Existence of proofs that a duplicate of the audiometric examination has been delivered and signed by the employee.	1.0
Does the service have qualified professionals to carry out educative actions?	Existence of a specialized and qualified professional in the field in question to carry out training programs.	1.0
Is there educational material in the company?	Existence of user’s, procedure, and norm manuals, booklets, and pamphlets	2.0
Are there minutes in the company of the educative actions?	Existence of minutes of the training and meetings carried out with the employees, presenting at least (1) date, (2) content approached, and (3) signature of the employees that participated.	1.0
Does the service have an assessment instrument of the Hearing Conservation Program?	Existence of an assessment instrument of the Hearing Conservation Program. It can be audit protocols, checklists, and/or assessment questionnaires.	2.0
TOTAL EXPECTED/OBTAINED SCORES IN THE ASSESSMENT OF RESOURCES		55.0

SET II - ASSESSMENT OF THE PROCESSES OF THE HEARING CONSERVATION PROGRAM
RISK ANALYSIS AND PLANNING
EVALUATIVE QUESTION	EXPECTED STANDARD/NORM	EXPECTED SCORE	SCORE OBTAINED
Have the responsibilities, goals, and priorities of the Hearing Conservation Program been defined or are they being defined?	Responsibilities, goals, and priorities in the period encompassing the implementation of the Hearing Conservation Program and/or its annual monitoring.	3.0
Is a preliminary assessment of noise exposure at the company conducted/has it been conducted?	Characterization of the risk exposure, applied to all the operational and habitual conditions of the employees when performing their duties, either annually or whenever there is a change in the environment.	3.0
Have homogeneous groups with similar exposure characteristics been identified/are they identified?	Identification of homogeneous groups with similar exposure characteristics, considering all those exposed in the group in question or covering one or more employees whose situation corresponds to the “typical” exposure of each group in question.	2.0
Is there/has there been an analysis of (co)existence of other risk agents (chemical, physical, and biological)?	Recognition and assessment of the employees’ exposure to chemical, physical, and/or biological agents that might trigger an occupational hearing loss.	2.0
Is the noise exposure in the company quantitatively assessed/has it been assessed?	Obtention of an estimate of the employees’ exposure, according to (1) the established daily dose of noise exposure, considering 100% as the limit of daily occupational continuous or intermittent noise exposure and 50% as the daily dose for the action level of occupational noise exposure; (2) the established exposure level and normal exposure level, considering his one equal to 80 dB(A); (3) the established correlation between maximum admissible peak levels and the number of impacts occurred during the workday, considering the peak level of 140 dB(Lin) as the tolerance limit of impact noise.	3.0
Have the action strategies and methodologies of the Hearing Conservation Program been established/are they being established?	Established action strategies and methodologies and the annual schedule or the one regarding the time encompassed in the current implementation of the Hearing Conservation Program.	3.0
Is the annual planning of the Hearing Conservation Program presented to and discussed with the departments involved in it/has it been presented and discussed?	Yearly presentation of the annual planning of the Hearing Conservation Program to the departments involved in it, having it discussed with them.	2.0
ENVIRONMENTAL CONTROL
EVALUATIVE QUESTION	EXPECTED STANDARD/NORM	EXPECTED SCORE	SCORE OBTAINED
Is there/has there been participation in the implementation of the engineering measures?	Participation on the part of the administrator of the Hearing Conservation Program, along with the person or team responsible for it, in the implementation of intervention measures (1) on the issuing source (changes in or replacement of equipment, machinery, and tools); and/or (2) implementation of measures to reduce the sound pressure level in the transmission (isolating or muffling vibrating surfaces, reducing the reverberation, adjustment or improvement in the preventive maintenance, changes in the paces and processes of operation, conception, and changes of the layout of the work settings).	2.0
Is there/has there been participation in the implementation of administrative measures?	Participation on the part of the administrator of the Hearing Conservation Program, along with the administrator of the company/organization, in the implementation of administrative measures, involving (1) changes in the operations; and/or (2) changes in the work procedures; and/or (3) employee’s leave of absence from the workplace or the risk source.	2.0
Is the personal protective equipment properly selected/has it been properly selected?	Selection of hearing protection devices technically adequate to (1) the characteristics of the environment and activity; (2) characteristics of the user; (3) characteristics of hearing protection; (4) required noise reduction level; (5) noise reduction level of the protection; (6) exposure level with the protection; (7) annual personal attenuation review, or whenever the model or size is changed.	2.0
Is the personal protective equipment supplied to the employees and replaced/has it been supplied and replaced?	Supply of hearing protection devices approved by the national department responsible for occupational safety and health and immediate replacement of the equipment when it is damaged or lost.	2.0
Is there/has there been an inspection of the use of the personal protective equipment on the part of the employees?	Inspect the use of personal protective equipment on the part of the employees, making use of (1) monitoring protocol of the use of hearing protection and the necessary periodicity, and (2) communicate to the Ministry of Labor and Employment any irregularity observed.	2.0
Have the employees who are entitled to a hazard pay or special retirement due to noise exposure defined/are they defined?	Aid the team to define the employees who must receive a hazard pay or special retirement due to noise exposure and refer them to Social Security.	2.0
HEARING HEALTH ADMINISTRATION
EVALUATIVE QUESTION	EXPECTED STANDARD/NORM	EXPECTED SCORE	SCORE OBTAINED
Are verification and control procedures regarding the functioning of the audiometer periodically conducted/have they been conducted?	(1) Submit the audiometer to annual electroacoustic verification, in compliance with norm ISO 8253-1; (2) Perform biological verification of the audiometer(s) before the audiometric examinations.	3.0
Are the employees submitted to occupational anamnesis/have they been submitted to it?	Clinical occupational anamnesis conducted in the audiometric assessment, in all types of examinations (pre-employment, sequential, post-leave of absence, and post-employment examinations).	2.0
Are the employees’ ears examined (meatoscopy)/have they been examined?	Examination of the external acoustic meatus of both ears by the person responsible for the audiometric examination, (1) when performing an audiometric assessment, in all types of examinations (pre-employment, sequential, post-leave of absence, and post-employment examinations) and (2) writing the findings in the audiometric examination form.	2.0
Are all the employees submitted to the pre-employment audiometric examination/have they been submitted to it?	Submitting the employees to pre-employment audiometric examination.	3.0
Are the audiometric examinations classified as a reference?	Establishing the reference audiometric examination when the employee does not have one or whenever an occupational hearing loss is triggered or worsened.	3.0
Are sequential audiometric examinations conducted/have they been conducted?	Conducting sequential audiometric examinations (1) in the 6th (sixth) month after employment or based on the reference audiometric examination; (2) annually, beginning in the 6th (sixth) month after employment.	3.0
Are post-employment audiometric examinations conducted/have they been conducted?	Conducting audiometric examination when the employee is dismissed, except for the situations provided in the Regulatory Norm no. 7.	3.0
Is speech audiometry conducted/has it been conducted?	Conducting speech audiometry (Speech Recognition Percentage, Speech Recognition Threshold, and/or Voice Detection Threshold) in the pre-employment examination and/or when occupational hearing loss is triggered.	2.0
Are they referred to complementary audiological examinations/have they been referred?	Performing other complementary audiological examinations, required by the physician who coordinates the Occupational Health Medical Control Program, to have a differential diagnosis.	2.0
Are they referred to specialized assessment/have they been referred?	Referring to other medical specialties or sectors, if necessary, when non-occupational hearing loss occurs.	2.0
Are criteria established/have they been established for a differential diagnosis of occupational/non-occupational hearing losses?	Referring the patient to the physician who coordinates the Occupational Health Medical Control Program or the one responsible for it to conduct the medical examination or, if they are absent, to the physician who assists the employees.	2.0
Is the evolution of hearing loss analyzed and diagnosed/has it been analyzed and diagnosed?	Referring the employee to the physician who coordinates the Occupational Health Medical Control Program to (1) analyze the evolution and define the evolution diagnosis of all the examinations performed, classifying the results as “normal hearing”, “occupational hearing loss”, or “non-occupational hearing loss”; (2) characterize the causal nexus between auditory damages and environmental risks, to guide the environmental control measures.	3.0
Are the results coming from the audiological diagnosis administration registered/have they been registered?	Including the cases suggestive of hearing loss induced by high sound pressure levels in the annual report of the Occupational Health Medical Control Program.	2.0
Are accidents at work communicated/have they been communicated?	Communicating the accidents at work suffered by employees with occupational hearing loss.	3.0
Are the employees given a duplicate of the examinations and certificates/have they been given them?	Making available (1) duplicates of audiometric examinations and (2) occupational health certificates to all the employees.	2.0
Are the records coming from the implementation of the program being administered and saved?	Maintaining, for at least 20 (twenty) years after the employee has been dismissed, the records of the results of the audiometric examinations, environmental assessments, and measures used for collective protection.	2.0
Are there/have there been training programs, debates, commissions, participation in events, and/or other appropriate forms involving the effects on health caused by the exposure to high sound pressure levels to all those involved in the Hearing Conservation Program?	Having carried out at least one of the actions in the previous 12 (twelve) months, involving the effects on health caused by the exposure to high sound pressure levels.	3.0
Are the employees trained about the implementation of collective and individual measures/have they been trained?	Having carried out at least one of the actions in the previous 12 (twelve) months, involving the procedures that ensure its effectiveness with hearing protection adjustment test, maintenance, replacement, hygiene, and information procedures regarding the possibly limited protection they provide.	3.0
Are the employees’ suggestions welcomed/have they been welcomed?	Welcoming and registering the employees’ suggestions brought up in meetings and debates.	2.0
ASSESSMENT
EVALUATIVE QUESTION	EXPECTED STANDARD/NORM	EXPECTED SCORE	SCORE OBTAINED
Are quantitative analyses of prevalence and incidence of the results of the audiological examinations carried out/have they been carried out?	Annually presenting statistics of the results of nosologic diagnoses (prevalence: normal hearing, occupational hearing loss, or non-occupational hearing loss) and evolution diagnoses (incidence: triggering or worsening of occupational hearing loss).	3.0
Is the employees’ degree of satisfaction assessed/has it been assessed?	Annually assessing the employees’ degree of satisfaction, based on their opinions regarding the implementation of the Hearing Conservation Program.	2.0
Is the degree of implementation of the Hearing Conservation Program being estimated with audits/has it been estimated?	Measuring the degree of implementation of the components of the Hearing Conservation Program systematically before, during, or after implementing the program, at least once a year.	3.0
Are the contextual factors that can influence the implementation of the Hearing Conservation Program being considered and analyzed/have they been considered and analyzed?	Conducting overall and contextual analyses, identifying the favorable or unfavorable factors to the ideal implementation of the Hearing Conservation Program in the company, relating them to the results of the audits.	2.0
TOTAL EXPECTED/OBTAINED SCORES IN THE ASSESSMENT OF PROCESSES		87.0

ANALYSIS AND INTERPRETATION OF THE RESULTS
FORMULA TO CALCULATE THE WEIGHTED MEAN OF THE ANSWERS TO OBTAIN THE DEGREE OF IMPLEMENTATION OF THE HEARING CONSERVATION PROGRAM $\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{}\mathrm{D}\mathrm{I}=\left(\frac{\left(4\sum S^1+6\sum P^1\right)∕10}{\left(4\sum S^2+6\sum P^2\right)∕10}\right)\mathrm{*}100$
Caption: Σ S¹ = Sum of the scores obtained in the indicators that made up the structure dimension. Σ P¹ = Sum of the scores obtained in the indicators that made up the process dimension. Σ S² = Sum of the scores expected for the indicators that made up the structure dimension. Σ P² = Sum of the scores expected for the indicators that made up the process dimension. Observation: The formula above represents the calculation of the weighted mean of the answers, considering the score obtained with the application of the instrument in relation to the total expected score. The items that make up the structure (resources) are multiplied by 4 (four), while the items that make up the processes (activities) are multiplied by 6 (six). The result obtained with the formula is presented in percentage.

CUTOFF SCORES
< 25.0%	Not Implemented
25.1% to 50.0%	Incipient Implementation
50.1% to 75.0%	Partially Implemented
> 75.1%	Implemented

PERCENTAGE OBTAINED IN THE DEGREE OF IMPLEMENTATION

RESULTS OF THE ASSESSMENT OF THE DEGREE OF IMPLEMENTATION OF THE HEARING CONSERVATION PROGRAM

Percentage obtained: ______ % Degree of Implementation of the Program: ___________________________________________ Aspects that need improvement: • ___________________________________________________________________________________________________   ___________________________________________________________________________________________________ • ___________________________________________________________________________________________________   ___________________________________________________________________________________________________ • ___________________________________________________________________________________________________   ___________________________________________________________________________________________________ • ___________________________________________________________________________________________________   ___________________________________________________________________________________________________ • ___________________________________________________________________________________________________   ___________________________________________________________________________________________________ • ___________________________________________________________________________________________________   ___________________________________________________________________________________________________ • ___________________________________________________________________________________________________   ___________________________________________________________________________________________________ Complementary observations of the assessor: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ Instructions/Suggestions: ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ ____________________________________________________________________________________________________ Date:____________________ Signature: ______________________________________

Publication Dates

• Publication in this collection
  10 Mar 2021
• Date of issue
  2021

History

• Received
  29 June 2020
• Accepted
  14 Jan 2021