Creation and content validity of a scale for assessing adherence to good practices for COVID-19

Construcción y validación de contenido de escala para evaluar adherencia a buenas prácticas para COVID-19

ABSTRACT

Objective:

to create a scale for assessing the adherence of healthcare providers to good practices for COVID-19 in the hospital setting and determine its content and face validity.

Methods:

A methodological study was developed in three sequential phases: creation of items and domains based on a literature review and administration of the questionnaire to 16 nursing professionals; assessment of content and face validity by experts; and semantic analysis.

Results:

The 51-item scale was submitted to the evaluation of seven experts. Items with a content validity index ≥ 0.83 were maintained. During the semantic analysis performed by the administration of the scale to 37 healthcare providers, no suggestions for changes were made and the comprehension rate was 87%.

Conclusions:

the 47-item instrument with three dimensions (personal, organizational, and psychosocial) achieved satisfactory content and face validity, meeting the parameters established in the literature.

Descritptors:
Validation Study; Coronavirus Infection; Protocols; Nursing; Reproducibility

RESUMEN

Objetivo:

construir y validar una escala para evaluar la adherencia de los profesionales de la salud a las buenas prácticas para el COVID-19 en el hospital y determinar su contenido y validez aparente.

Métodos:

estudio metodológico, desarrollado en tres fases secuenciales: creación de ítems y dominios a partir de revisión bibliográfica y aplicación de cuestionario a 16 profesionales de enfermería; evaluación de contenido y validez aparente por especialistas; y análisis semántico.

Resultados:

la escala de 51 ítems fue sometida a evaluación por siete expertos. Se mantuvieron los ítems con índice de validez de contenido ≥ 0,83. Durante el análisis semántico realizado por la administración de la escala a 37 proveedores de salud, no se hicieron sugerencias de cambios y la tasa de comprensión fue del 87%.

Conclusiones:

el instrumento con 47 ítems y tres dimensiones (personal, organizacional y psicosocial) alcanzó la validez de contenido y apariencia con un estándar satisfactorio, además de alcanzar los parámetros recomendados por la literatura.

Descriptores:
Estudio de Validación; Infecciones por Coronavirus; Protocolos; Enfermería; Reproducibilidad de los Resultados

RESUMO

Objetivo:

Construir e validar conteúdo e face de escala para avaliar a adesão dos profissionais de saúde às boas práticas para COVID-19, no âmbito hospitalar.

Métodos:

Estudo metodológico, desenvolvido em três fases sequenciais: construção dos itens e dimensões a partir de revisão de literatura e aplicação de questionário a 16 profissionais de enfermagem; validação de conteúdo e de face por especialistas e análise semântica.

Resultados:

A escala com 51 itens foi submetida à avaliação de sete especialistas e foram mantidos os itens que apresentaram o índice de validade de conteúdo ≥0,83. Na análise semântica, realizada mediante aplicação da escala a 35 profissionais de saúde, não houve sugestão de modificação e o percentual de compreensão foi de 0,87%.

Conclusões:

O instrumento com 47 itens e três dimensões (pessoal, organizacional e psicossocial), alcançou a validade de conteúdo e de face com padrão satisfatório, além de atingir os parâmetros preconizados pela literatura.

Descritores:
Estudo de Validação; Infecções por Coronavírus; Protocolos; Enfermagem; Reprodutibilidade dos Testes

INTRODUCTION

In December 2019, the first cases of a disease named COVID-19 (CO = Corona; VI = Virus; D = Disease) emerged in Wuhan, China, which were caused by the new coronavirus of severe acute respiratory syndrome 2 (SARS-CoV-2)(¹). Subsequently, declared a global emergency by the World Health Organization (WHO), the new coronavirus pandemic initiated in 2020, the characteristics of which are cough and fever that can progress to pneumonia and, in more severe cases, dyspnea and even death. The incubation period ranges from two to 14 days and, in some cases, individuals may be asymptomatic but considered vectors of transmission(²). Transmission is believed to occur mainly through droplets as well as aerosols (suspended particles) or contact between contaminated material and mucous membranes. An estimated 7 to 16% of all cases of COVID-19 require hospitalization, and 5 to 12% require intensive care(³-⁴). The WHO has emphasized the importance of adopting protection measures to avoid the transmission of the disease, such as distancing between individuals, use of a mask over the nose and mouth to contain respiratory secretions, identification and isolation of individuals with suspected or confirmed infection, use of personal protective equipment and sanitization. At healthcare services that provide care for individuals with COVID-19, adequate standard precautions are essential, such as hand sanitizing and use of personal protective equipment (PPE)(⁵) (gloves, masks, protective eyewear and isolation gown), whenever the risk of being exposed to blood or other organic fluids occurs(⁶).

Besides the use of PPE, it is also important to consider the order of putting on and taking off garments, as the risk of indirect contamination is greater when taking off garments.(⁷) To avoid hospital transmission, all prevention measures for COVID-19 should be followed, ideally with the institution of protocols and the training of health teams. It is therefore important to determine whether such protocols are indeed being followed. For such, tools can be applied that address adherence to the precaution measures instituted and enable the identification of difficulties in the use of such measures.

Adherence is a dynamic, multifactorial process that results from a set of determinants dependent on subjective, organizational and work-related factors. It is a complex decision-making process mediated by psychological traits, intellectual/cognitive level, beliefs, values and sociocultural context(⁸).

However, assessing adherence to precautions for COVID-19 became a challenge during the pandemic, as few instruments are available in the literature that enable the assessment of adherence to standard precautions and none include specificities inherent to COVID-19 transmission. These aspects are generally more directed at other diseases(⁹-¹⁰). Thus, the present study was guided by the following research question: what is the content and face validity of a novel instrument developed to assess the adherence of healthcare providers to good hospital practices for COVID-19?

OBJECTIVE

Create a scale for assessing the adherence of healthcare providers to good practices for COVID-19 in the hospital setting and determine its content and face validity.

METHODS

Ethical aspects

This study received approval from the Human Research Ethics Committee of the Faculdade de Medicina de Botucatu, Universidade Estadual Paulista.

Study design, setting and period

A methodological study was developed for the creation of a scale to assess the adherence of healthcare providers to good practices for dealing with COVID-19. The study was conducted between April and October 2020. The design involved the following steps: construction of items based on a literature review; content/face validation and semantic analysis.

This study was developed at the hospital affiliated with the Faculdade de Medicina de Botucatu of Universidade Estadual Paulista; and was structured according to COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) (¹¹).

Step 1 - Creation of scale items

The creation of the instrument began with a review of the literature in the US National Library of Medicine (PubMed Central), Web of Science, Latin American and Caribbean Health Sciences Literature (LILACS) and Current Nursing and Allied Health Literature (CINAHAL) databases, which included publications as well as catalogues of theses, protocols and dissertations. A search was also conducted of the sites of governmental and non-governmental agencies, such as the World Health Organization (WHO), Pan America Health Organization (PAHO), Centers for Disease Control and Prevention (CDC) and Agência Nacional de Vigilância Sanitária (ANVISA [Brazilian National Health Surveillance Agency]), including national and international protocols. However, all articles found at the beginning of the pandemic regarded the origin, transmission, diagnosis and treatment of COVID-19 as well as clinical cases and experience reports. In this search, only one study was found that addressed the role of nurses in the management of patients with COVID-19 based on a literature review on the subject(¹²).

The Health Belief Model, developed by psychologists of the United States in the 1990s, was used as theoretical framework. This model was developed in an attempt to explain why certain people do not correctly prevent some diseases for which there are tests or vaccines. The model is composed of four components: perceived susceptibility; severity; benefits; and barriers. Stimuli that lead to decision-making are also part of the model, which may have an internal or external origin(¹³). According to the theoretical model, the adoption of preventive behavior depends greatly: on whether one considers oneself vulnerable to a particular health problem and believes that the problem could affect one at some point (perceived susceptibility); whether one perceives that a health problem could have serious consequences (perceived severity); and whether one believes that the health problem can be prevented by a given action (perceived benefits), regardless of whether such action involves negative aspects (perceived barriers), meaning that action’s beliefs are assessed as a function of the barriers to performing it(¹⁴).

From the bibliographic survey of the subject, eleven open-ended questions were created, which were subsequently sent via e-mail (due to the impossibility of forming a focus group during the pandemic) to nine nurses and seven nursing technicians who had been asked to participate in the study and worked at the institution where the study was developed. At the time, other healthcare providers were also invited, but declined to participate due to excessive workload and time unavailability to answer the questions. The answers were analyzed using the content analysis method proposed by Bardin(¹⁵), which consists of three phases: pre-analysis; material exploration; and treatment of results and interpretation.

Step 2 - Content and face validation of items

The judges were selected based on their work as researchers, experts in Intensive Care Units (ICU) and/or emergency rooms and and/or participation on health-associated infection boards. The initial invitation to the group of judges was formal and occurred through a letter sent by e-mail containing the study objectives, the instrument description, information on completing the instrument, the reason why the individual was chosen to be a judge(¹⁶) and the statement of informed consent. Some questions were also asked aiming at characterizing the participants and the new scale.

Twelve physicians and nurses who met the selection criteria for the study were invited to participate and given a period of 15 days for content analysis, which was extended for another 15 days due to the pandemic. However, only seven of the instruments were returned answered. The scientific literature recommends that the number of experts selected vary between two and ten individuals(¹⁷). Thus, seven are considered adequate for agreement assessment(¹⁸).

Each item on the instrument was scored from one to five points on a Likert scale with regards to clarity, relevance, representativeness and comprehensiveness (not relevant [1 point]; requires major revision [2 points]; requires minor revision [3 points]; and relevant [4 points])(¹⁶). The description of these concepts was included in the orientations sent to the judges, in accordance with the literature framework adopted for developing scales(¹⁹). For each item, a space was allotted for the judges to offer suggestions and/or changes. An assessment option was also offered for the dimension categorization: personal, organizational, and psychosocial.

However, due to the large number of suggestions from the judges regarding changes to the writing of several items and item separation related to removing garments, a second round of content analysis was performed with the same judges who developed the instrument. The content validity index (CVI) was calculated by the sum of agreement of items scored as “3” or “4” in the assessment by the judges, divided by the total number of answers(²⁰) in relation to each criterion. An acceptable agreement rate among judges for item validity was established as > 0.83 due to the number of judges, as recommended in the literature(¹⁷).

Step 3 - Semantic analysis

The purpose of semantic analysis is to assess whether all items are adequate and understandable to the population for which the instrument was designed and should be administered to 30 to 40 individuals(²¹). For the selection of this sample, 40 healthcare providers who worked in COVID-19 wards, emergency care and adult ICUs were chosen between July and August 2020. Thirty-seven of these professionals agreed to participate.

RESULTS

A 51-item instrument was initially sent for assessment by 12 experts, seven of whom responded within 30 days. Most of these judges were women (85.7%) and a large portion had more than 16 years of professional experience (42.9%). Regarding academic background, five were nurses and one was a physician. Three were on health-related infection boards at public and private institutions (42.9%), one worked in an ICU (14.2%) and the others were involved in teaching and research (42.9%).

Due to the large quantity of changes suggested for the items during the first assessment and separation of one item, a second round of assessment was performed with the same experts. The following were the main suggestions for change: writing in the first person, transformation of negative wording into positive wording, change of title, correction of terms and terminology, item separation to removing garments and the replacement of the word SARcov-2 with COVID-19.

The initial title “Scale of Adherence to Standard Precautions for COVID-19” was changed to “Scale of Adherence to Good Hospital Practices for COVID-19”. In this first round, two items were excluded due to experts’ negative assessment, which unanimously considered the items to have little relevance. The first item referred to the use of an inhalation mask or oxygen catheter for patients with no indication for mechanical ventilation. The second item referred to the need for terminal cleaning of the bathroom after an aspersion bath for patients with suspected or confirmed COVID. A third item was also excluded for being duplicated. At this point, the scale had been reduced to 48 items.

As one of the suggestions was the item separation addressing the removal of garments, considering that it should be initiated in the room and finished outside the room. There was the inclusion of an additional item, leading to a 49-item scale. The instrument was then submitted to a second assessment round by the same experts for calculating the CVI. All suggested changes were made prior to semantic analysis by the target population. Only items 5, 17, 22, 27, 28, 31, 40, 46, 47 and 51 were not changed (Chart 1).

Chart 1
Changes made to items of instrument after assessment by expert committee, Botucatu, São Paulo, Brazil, 2020

Initial item	1st assessment	2nd assessment
1. Performs hand sanitizing with bar soap and water (40-60 seconds) or 70% alcohol (20-30 seconds)	Performs hand sanitizing with liquid soap and water (40-60 seconds) or a 70% alcohol solution (20-30 seconds)	I perform hand sanitizing with liquid soap and water (40-60 seconds) or a 70% alcohol solution (20-30 seconds)
2. Before entering the isolation room, adequate conditions for hand washing or a dispenser with a 70% alcohol preparation, trashcan with a lid and opening without hand contact are always available	Before entering the room with precaution, adequate conditions for hand sanitizing and/or a dispenser with a 70% alcohol solution, trashcan with a lid and opening without hand contact are always available	Before entering the room/box with precaution, adequate conditions for hand sanitizing and/or a dispenser with a 70% alcohol solution, trashcan with a lid and opening without hand contact are always available
3.I feel safe in the work environment in this pandemic situation	I feel safe in the work environment in this pandemic situation	I feel safe in the work environment in this pandemic situation
4. All professionals receive institutional orientations on how to use, remove and discard PPE adequately	I receive orientations from the institution on how to use, remove and discard PPE adequately	I received orientations from the institution on how to use, remove and discard PPE adequately
5. PPE is NOT removed and discarded as infectious waste in an appropriate location following a procedure and before leaving the patient’s room or isolation area	I remove PPE and discard it as infectious waste in an appropriate location following a procedure and before leaving the patient’s room or isolation area	I remove PPE and discard it as infectious waste in an appropriate location following a procedure and before leaving the patient’s room/box or isolation area
6. Stress at work interferes with interpersonal relations in a negative way	Stress at work interferes with my professional performance in providing care to patients with a suspicion or confirmation of SAR-Cov2 in a negative way	Stress negatively interferes with my professional performance in providing care to patients with a suspicion or confirmation of COVID-19
7. My schedule respects the break time established between shifts so that I can get adequate rest	My work schedule respects the break time established between shifts so that I can get adequate rest	My work schedule respects the break time established between shifts so that I can have adequate rest
8. Uses some type of PPE regardless of suspicion or confirmation of patients with SARS-Cov-2	Uses some type of PPE (mask, gown, protective eyewear, etc.) regardless of suspicion or confirmation of patients with SARS-Cov-2	I use some type of PPE (mask, gown, protective eyewear, etc.) regardless of suspicion or confirmation of patients with COVID-19
9. Considers all materials that were in contact with patients contaminated even without a confirmation of the diagnosis of SARS-Cov-2	Considers all materials contaminated that were in contact with patients with or without a confirmation of the diagnosis of SARS-Cov-2	I consider all materials contaminated that were in contact with patients with or without a confirmation of the diagnosis of COVID-19
10. Stethoscopes, sphygmomanometer, thermometers, bedpans and urine pots are not the exclusive use of patients with a suspicion or confirmation of infection by SARS-Cov-2	Stethoscopes, sphygmomanometer, thermometers, bedpans and urine pots are the exclusive use of patients with a suspicion or confirmation of infection by SARS-Cov-2	Stethoscopes, sphygmomanometer, thermometers, bedpans and urine pots are the exclusive use of patients with a suspicion or confirmation of COVID-19
11. I try to keep my thoughts positive during the entire shift	I try to keep my thoughts positive during the entire shift to follow orientations with regards to the use of PPE	I try to keep my thoughts positive during the entire shift to follow orientations with regards to the correct use of PPE
12. In rooms that do not have private bathrooms, the preference is a sponge bath in bed even for patients who are conscious and can walk	In rooms that do not have private bathrooms, the preference is a sponge bath in bed or the use of a bathroom exclusively limited to patients with COVID-19	In rooms that do not have private bathrooms, the preference is a sponge bath in bed or the use of a bathroom exclusively limited to patients with COVID-19
13. The principles of standard precautions for infection control and precautions based on transmission continue being followed for the handling of the body even after death	The principles of the use of standard precautions for the control of infection and transmission for COVID-19 continue being followed for the handling of the body even after death	The principles of the use of standard precautions for the control of infection and transmission of COVID-19 continue being followed for the handling of the body even after death
14. I feel my energy drained at the end of the shift	I feel my energy drained at the end of the shift and this compromises the adequate use of PPE	I feel my energy drained at the end of the shift and this compromises the adequate use of PPE during the pandemic
15. After death, orifices (oral, nasal and rectal) are plugged/blocked still in the unit to avoid the leaking of bodily fluids when transporting the body	After death, orifices (oral, nasal and rectal) are plugged/blocked still in the unit to avoid the leaking of bodily fluids when transporting the body	After death, orifices (oral, nasal and rectal) are plugged/blocked still in the unit to avoid the leaking of bodily fluids when transporting the body to avoid the transmission of COVID-19
16. Lately, I have felt fear regarding any possibility of the risk of contagion	Lately, I have felt fear regarding any possibility of the risk of contagion by COVID-19	Lately, I have felt fear regarding any possibility of the risk of contamination by COVID-19
17. Diapers are offered to patients who are in isolation without a private bathroom even if they can walk	For patients who are in isolation for the prevention of the transmission of COVID-19 and who do not have a private bathroom, diapers are offered even to those who can walk	For patients who are in isolation for the prevention of the transmission of COVID-19 and whose room does not have a private bathroom, diapers are offered even to those who can walk
18. I change my mask whenever it is moist and dirty	I change my facial mask whenever it is moist, dirty or damaged	I change my facial mask whenever it is moist, dirty or damaged
19. I am afraid to go back home and imagine that I could be transmitting this disease to my family	I am afraid to go back home and imagine that I could be transmitting COVID-19 to my family	I am afraid to go back home and imagine that I could be transmitting the new coronavirus to my family
20. I perform the positive and negative seal test of the mask on my face and discard the mask immediately if the test is not successful	I perform the positive and negative seal test of the N95 or similar mask on my face and discard the mask immediately if the test is not successful	I perform the positive and negative seal test of the mask on my face and, when the test is not successful, I discard the mask immediately
21. The institution does not provide surgical masks for symptomatic patients (cough, sneezing, etc.) at the entrances of the healthcare services, waiting room, etc.	The institution provides surgical masks for symptomatic patients (cough, sneezing or other cold signs or symptoms) at the entrances of the healthcare services, waiting room, etc.	The institution provides surgical masks for symptomatic patients with cough, sneezing or other cold signs or symptoms at the entrances of the healthcare services, waiting room, etc.
22. My head is filled with worry about SARS-Cov-2	The insecurity caused by the possibility of contamination by COVID-19 interferes with my adherence to the use of PPE	The insecurity caused by the possibility of contamination by COVID-19 interferes with my adherence to the use of PPE
23. During invasive procedures in patients infected by SARS-Cov-2, the presence of a nursing assistant remaining outside the isolation area to fulfill the requests of the team is not necessary	During invasive procedures in patients infected by or suspected of having SARS-Cov-2, the presence of a nursing assistant remaining outside the isolation area to fulfill the requests of the team is necessary	During invasive procedures in patients infected or suspected of having COVID-19, the presence of a nursing assistant outside the isolation area to fulfill the requests of the team is necessary
24. I use an N95 or equivalent respiratory protection mask in procedures with the risk of the generation of aerosols	I use an N95/PFF2 or equivalent respiratory protection mask in procedures with the risk of the generation of aerosols	I use an N95/PFF2 or equivalent respiratory protection mask in procedures with the risk of the generation of aerosols
25. Suspected or confirmed cases of infection by SARS-Cov-2 are lodged in a private, well-ventilated (windows open) room with an exclusive bathroom and the door closed	Suspected or confirmed cases of infection by SARS-Cov-2 are lodged in a private, well-ventilated (windows open) room with an exclusive bathroom and the door closed	Suspected or confirmed cases of infection by COVID-19 are lodged in a private, well-ventilated (windows open) room with an exclusive bathroom and the door closed
26. I consider myself capable of getting through this pandemic situation without becoming contaminated in the work environment	I consider myself capable of getting through this pandemic situation without becoming contaminated in the work environment	I consider myself capable of getting through this pandemic situation without becoming contaminated in the work environment by my faith
27. All materials for invasive procedures are prepared outside of the isolation area	For patients with COVID-19, all materials for invasive procedures are organized outside of the isolation area	For patients with COVID-19, all materials for invasive procedures are organized/prepared outside of the room or isolation area
28. After leaving the room or isolation area, I remove garments in the following order: I sanitize my hands; I remove the gown; I remove the protective eyewear or facial protector; Next, I sanitize my hands again; I remove the mask and, lastly, I sanitize my hands	I remove garments in the following order when still in the room: I remove the gloves and gown and sanitize my hands	I remove garments in the following order when still in the room/box: I remove the gloves and gown and sanitize my hands
29. I do not use PPE adequately due to the excessive workload	I do not use PPE adequately due to the excessive daily workload during the pandemic	I do not use PPE adequately due to the excessive daily workload during the pandemic
30. For patients with SARS-CoV-2 on mechanical ventilation, only the closed aspiration system is performed	For patients with SARS-CoV-2 on mechanical ventilation, the closed aspiration system is preferably used	For patients with COVID-19 on mechanical ventilation, the closed aspiration system is preferably used
31.When changing the bedding, I follow the orientation to handle it as little as possible to avoid agitation	When changing the bedding, I follow the orientation to handle it as little as possible to avoid the dispersion of particles	When changing the bedding, I follow the orientation to handle it as little as possible to avoid the dispersion of particles
32. The service provides gowns of different types depending on the clinical condition of the patient. Ex: water-proof gown in cases of vomiting, diarrhea, orotracheal hypersecretion, bleeding, etc.	The institution provides gowns of different types depending on the clinical condition of the patient. Ex: water-proof gown in cases of vomiting, diarrhea, orotracheal hypersecretion, bleeding, etc.	The institution provides gowns of different types depending on the clinical condition of the patient. Ex: water-proof gown in cases of vomiting, diarrhea, orotracheal hypersecretion, bleeding, etc.
33. I feel guilty when some undesired event outside of my control occurs on my shift	I feel guilty when some undesired event outside of my control occurs on my shift, such as a death	I feel guilty when some undesired event outside of my control occurs on my shift, such as a death
34. Waste from care provided to patients with a suspicion of or confirmation of SARS-Cov-2 is placed in a red bag or, in the absence of a red bag, in milk white bags with the infectious material symbol	Waste from care provided to patients with a suspicion of or confirmation of SARS-Cov-2 is placed in a red bag or, in the absence of a red bag, in milk white bags with the infectious material symbol	Waste from care provided to patients with a suspicion of or confirmation of COVID-19 is placed in a red bag or, in the absence of a red bag, in milk white bags with the infectious material symbol
35. The institution does not provide private clothing for working in COVID areas	The institution provides private clothing for working exclusively in COVID areas	The institution provides private clothing for working exclusively in areas of patients with COVID-19
36. I do not feel influenced by the opinions of my work colleagues	I feel influenced by the opinions of my work colleagues	I feel influenced by the opinions of my work colleagues
37. A minimum of 1 meter of distance is maintained between the beds of patients with a suspicion or confirmation of SARS-Cov-2 when there is no possibility of their being isolated individually	A minimum of one meter of distance is maintained between the beds of patients with a suspicion or confirmation of SARS-Cov-2 when there is no possibility of their being isolated individually	A minimum of one meter of distance is maintained between the beds of patients with a suspicion or confirmation of COVID-19 when there is no possibility of their being isolated individually
38. During work, I do not use accessories (rings, bracelets, watch), as microorganisms accumulate under these objects, which are not removed by washing the hands	During work, I do not use adornments (rings, bracelets, watch), as microorganisms accumulate under these objects, which are not removed by sanitizing the hands	I do not use adornments (rings, bracelets, watch) when providing care for patients with a suspicion or confirmation of COVID-19
39. I feel tense in the workplace	I feel tense when providing care to patients with a suspicion or confirmation of COVID-19	I feel tense when providing care to patients with a suspicion or confirmation of COVID-19
40. The institution provides training for all professionals (in-house or outsourced) for the prevention of the transmission of SARS-CoV-2	The institution provides training at the service for all professionals (in-house or outsourced) for the prevention of the transmission of SARS-CoV-2A	The institution provides training at the service for all professionals (in-house or outsourced) for the prevention of the transmission of COVID-19
41. I place garments in the following order: washing of hands, placement of head cover, washing of hand, placement of gown, mask, protective eyewear or face shield and gloves	I place garments in the following order: I sanitize my hands; I put on a long-sleeve gown, mask, protective eyewear or face shield and gloves	When putting on garments to provide care for patients with COVID-19, I perform hand sanitizing, followed by the placement of a disposable gown, protective eyewear or face shield, N95 or similar mask, long-sleeve gown and procedure gloves
42. I seek support and strength in God	I seek support and strength in God to continue providing my care to patients with a suspicion or confirmation of COVID-19	I seek support and strength in God to continue providing my care to patients with a suspicion or confirmation of COVID-19
43. I sanitize my hands five established times: before touching the patient, before performing a procedure, after risk of exposure to fluids, after the procedure and after touching surfaces close to the patient	I sanitize my hands five established times: before touching the patient, before performing a procedure, after risk of exposure to fluids, after the procedure and after touching surfaces close to the patient	I sanitize my hands five established times: before touching the patient, before performing a procedure, after risk of exposure to fluids, after the procedure and after touching surfaces close to the patient
44. Health professionals who provide care directly to patients with a suspicion or confirmation of infection by SARS-Cov-2 are scheduled to work only in this area during their entire work shift and do not circulate in other areas that provide care	Health professionals who provide care directly to patients with a suspicion or confirmation of infection by SARS-Cov-2 are scheduled to work only in this area during their entire work shift and do not circulate in other areas that provide care	Health professionals who provide care directly to patients with a suspicion or confirmation of infection by COVID-19 are scheduled to work only in this area during their work shift and do not circulate in other areas that provide care
45. I use protective eyewear or a face shield to provide care even with no confirmation of the diagnosis	I make use of protective eyewear or a face shield to provide care for patients with a confirmation or suspicion of COVID-19	I make use of protective eyewear or a face shield to provide care for patients with a confirmation or suspicion of COVID-19
46. Uses standard precautions with all patients regardless of the confirmation of SARS-Cov-2	I use standard precautions with all patients regardless of the confirmation of SARS-Cov-2	I use standard precautions with all patients regardless of the confirmation of COVID-19
47. Upon removing the N95 or similar mask, it is discarded if I touch its inner surface	Upon removing the N95 or similar mask, it is discarded if I touch its inner surface	Upon removing the N95 or similar mask, it is discarded if I touch its inner surface
48. Item included after judges’ evaluation:	I remove garments in the following order upon leaving the room: I sanitize my hand, I remove the gown, eyewear or face shield; I sanitize my hands; I remove or maintain the N95 mask; and lastly, I sanitize my hands	After providing care for patients with COVID-19, I remove garments in the following order upon leaving the room/box: I sanitize my hand, I remove the gown, eyewear or face shield; I sanitize my hands; I remove the N95 mask; and lastly, I sanitize my hands
49. I do not believe that I may become contaminated by the new coronavirus in the workplace	I do not believe that I may become contaminated by the new coronavirus in the workplace	I do not believe that I may become contaminated by the new coronavirus in the workplace

Table 1 displays the CVIs calculated after the second assessment of the instrument by an expert committee and committee classification regarding the items that composed the three dimensions. A satisfactory CVI was found for all criteria assessed (CVI ≥ 0.94). However, item 36 had an unsatisfactory assessment with regard to clarity and comprehensiveness (CVI = 0.57), and item 47 had an unsatisfactory assessment with regard to clarity (CVI = 0.71), which led to the exclusion of these items. The representativeness criterion had the highest score among experts (CVI = 0.99).

Table 1
Content validity index of items in relation to criteria analyzed and dimension proposed by expert committee, Botucatu, São Paulo, Brazil, 2020

Items	1	2	3	4
Dimension: personal factors
1. I perform hand sanitizing with liquid soap and water [...]	1.00	1.00	1.00	1.00
5. I remove PPE and discard it as infectious waste [...]	1.00	1.00	1.00	1.00
8. I use some type of PPE (mask, gown, protective eyewear, etc.) [...]	0.86	0.86	0.86	1.00
9. I consider all materials contaminated that were [...]	1.00	1.00	0.86	1.00
14. I feel my energy drained at the end of the shift [...]	1.00	1.00	1.00	1.00
18. I change my facial mask whenever it [...]	1.00	1.00	1.00	1.00
20. I perform the positive and negative seal test [...]	1.00	0.86	1.00	1.00
24. I use an N95/PFF2 or equivalent respiratory protection mask in procedures with the risk of the generation of aerosols
28. I remove garments in the following order when still in the room/box [...]	1.00	1.00	1.00	1.00
36. I feel influenced by the opinions of my work colleagues	1.00	0.57	0.57	1.00
38. I do not use adornments (rings, bracelets, watch) when providing care [...]	1.00	1.00	1.00	1.00
43. I sanitize my hands five established times: before touching the patient, before performing a procedure, after risk of exposure [...]	1.00	1.00	1.00	1.00
46. I use standard precautions with all patients regardless [...]	1.00	0.86	0.86	1.00
49. I do not believe that I may become contaminated by the new coronavirus [...]	1.00	0.86	0.86	1.00
Dimension: Organizational factors
2. Before entering the room/box with precaution, adequate conditions [...]	1.00	1.00	1.00	1.00
4. I received orientations from the institution on how to use, remove [...]	1.00	1.00	1.00	1.00
7. My work schedule respects the break time established [...]	1.00	0.86	1.00	1.00
10. Stethoscopes, sphygmomanometer, thermometers, bedpans [...]	1.00	1.00	1.00	1.00
12. In rooms that do not have private bathrooms, the preference is [...]	0.86	0.86	0.86	1.00
13. The principles of the use of standard precautions for the control [...]	1.00	1.00	1.00	1.00
15. After death, orifices (oral, nasal and rectal) are plugged/blocked still in the unit to avoid [...]	1.00	1.00	1.00	1.00
17. For patients who are in isolation for the prevention of [...]	1.00	1.00	1.00	1.00
21. The institution provides surgical masks for symptomatic patients with cough, sneezing or other cold signs or symptoms at the entrances [...]	1.00	1.00	1.00	1.00
23. During invasive procedures in patients infected or suspected [...]	1.00	1.00	1.00	1.00
25. Suspected or confirmed cases of infection by COVID-19 are lodged in a private, well-ventilated (windows open) room [...]	1.00	0.86	1.00	1.00
27. For patients with COVID-19, all materials for invasive procedures are organized/prepared outside of the room or isolation area	1.00	0.86	1.00	1.00
29. I do not use PPE adequately due to the excessive daily workload [...]	0.86	0.86	1.00	1.00
30. For patients with COVID-19 on mechanical ventilation, the closed aspiration system is preferably used	1.00	1.00	1.00	1.00
31. When changing the bedding, I follow the orientation to handle it as little as possible to avoid the dispersion of particles	1.00	1.00	0.86	1.00
32. The institution provides gowns of different types depending on the clinical condition of the patient. [...]	1.00	0.86	1.00	1.00
34. Waste from care provided to patients with a suspicion of or confirmation of COVID-19 [...]	1.00	1.00	1.00	1.00
35. The institution provides private clothing for working exclusively in areas of patients with COVID-19	1.00	1.00	1.00	1.00
37. A minimum of one meter of distance is maintained between the beds of patients with a suspicion or confirmation of COVID-19 [...]	1.00	1.00	1.00	1.00
40. The institution provides training at the service for all professionals (in-house or outsourced) for the prevention [...]	1.00	1.00	1.00	1.00
41. When putting on garments to provide care for patients with COVID-19, I perform hand sanitizing, followed by the placement of a disposable gown [...]	1.00	1.00	1.00	1.00
44. Health professionals who provide care directly to patients with a suspicion or confirmation of infection by COVID-19 [...]	1.00	1.00	1.00	1.00
45. I make use of protective eyewear or a face shield to provide care for patients with a confirmation or suspicion of COVID-19	1.00	1.00	1.00	1.00
47. Upon removing the N95 or similar mask, it is discarded if I touch its inner surface	0.86	0.71	0.86	0.86
48. After providing care for patients with COVID-19, I remove garments in the following order upon leaving the room/box: I sanitize my hands [...]	1.00	1.00	1.00	1.00
Dimension: Psychosocial factors
3. I feel safe in the work environment in this pandemic situation	1.00	0.86	0.86	1.00
6. Stress negatively interferes with my professional performance [...]	1.00	0.86	1.00	1.00
11. I try to keep my thoughts positive during the entire shift [...]	1.00	1.00	1.00	1.00
16. Lately, I have felt fear regarding any possibility [...]	0.86	0.86	0.86	0.86
19. I am afraid to go back home and imagine that I could be transmitting [...]	1.00	1.00	1.00	1.00
22. The insecurity caused by the possibility of contamination [...]	1.00	0.86	1.00	1.00
26. I consider myself capable of getting through this pandemic [...]	1.00	1.00	1.00	1.00
33. I feel guilty when some undesired event outside of my control occurs on my shift, such as a death	0.86	0.86	0.86	0.86
39. I feel tense when providing care to patients with a suspicion [...]	1.00	0.86	1.00	1.00
42. I seek support and strength in God to continue providing [...]	0.86	0.86	0.86	0.86
Total	0.98	0.94	0.96	0.99

Note: 1= Relevance; 2 = Clarity; 3 = Comprehensiveness; 4 = Representativeness.

Regarding the personal dimension, which comprised items 1, 3, 5, 8, 9, 11, 13, 14, 15, 18, 20, 24, 28, 31, 36, 38, 41, 43, 45, 46, 47, 48 and 49, the judges made the following changes: items 13, 15, 31, 41, 45, 47 and 48 were moved to the organization dimension and items 3 and 11 were moved to the psychosocial dimension. Accordingly, the organizational dimension, which comprised items 2, 4, 7, 10, 12, 17, 21, 23, 25, 27, 29, 30, 32, 34, 35, 37, 40 and 44, also received seven items from the personal dimension. The psychosocial dimension, which comprised items 6, 16, 19, 22, 26, 33, 39 and 42, received two items from the personal dimension (3 and 11). Thus, the scale had 25 items addressing organizational factors, 14 addressing personal factors and 10 addressing psychosocial factors.

After calculating the CVI, the instrument was administered to 37 healthcare providers for assessment of the understanding of each item. Thirty-five of these professionals returned the instrument: 21 nurses (60%), 12 nursing technicians (34.2%) and two physicians (5.71%). Thirty-two were women (91.4%) and about half (51.4%) had up to five years of experience in the field.

In this phase, no suggestions were made for changes to the items, and the comprehension rate was 87%. Thus, the final instrument comprised 47 items (after the exclusion of items 36 and 47) distributed among three dimensions and each item was scored on a five-point scale: (1) Never; (2) Rarely; (3) Sometimes; (4) Often; and (5) Always.

DISCUSSION

For the creation of the items of the instrument, a broad review of the literature on the subject was performed and consultations were held with nursing professionals. Some difficulties were encountered due to the different suggestions made by the experts and the lack of standardization in the information found in the literature, especially with regards to the removal of garments.

For this item, beside suggesting the separation of the garment removal steps that should occur inside and outside the room or isolation area of the patient, a revision of the sequence of the placement of PPE was also suggested. One of the experts suggested not including sanitizing the hands after the removal of the protective eyewear or visor and head cover. However, as this is one of the guidelines established by ANVISA(²²) for care in cases suspected or confirmed COVID-19 cases, the suggestion could not be incorporated. The international literature does not offer a norm for the garment placement sequence. The WHO only cites the items that should be part of individual precaution. The CDC states that there is more than one safe garment placement sequence, but all recommend the use of a visor, head cover, mask, gown and gloves(²³).

Another problem related to this item regarded N95 or similar mask. One expert suggested including the term “maintain the N95”, as the mask is not exchanged for every procedure due to the cost and the duration of its lifespan. The conclusion was that the items should follow what is established in national and international literature and should not be linked to institutional routines. The use of N95 mask should follow the Hospital Infection Control Committee recommendations(²²). The WHO recommends avoiding prolonged mask use, and healthcare providers should change the mask in the period stipulated by the manufacturer. However, prolonged use is considered feasible and safe when the mask is stored and handled adequately, when it has no visible signs of damage and maintains an adequate seal verified by a test(²⁴).

Some recommendations were included during the item creation process, such as care with a dead body. Although this procedure is not performed by healthcare providers at all institutions, it is an important aspect about which individuals who work in the hospital setting should have knowledge. The same occurred with the item addressing room ventilation. Although many institutions do not have windows due a central cooling system, this is a recommended good practice that contributes to diminishing the contamination of the environment and, therefore, is part of disease transmission control measures(²⁵).

Our greatest obstacle was the creation of items addressing psychosocial aspects, which encompass beliefs and values. Creating an item that represented or reflected the real impact of these aspects on adherence to precautions, which is addressed little in the literature, was a considerable challenge. At time of the pandemic, with an abrupt onset and fear permeating among all individuals, the inclusion of such items was considered important, as such aspects came up in the discourse. Some instruments available in the literature for the assessment of adherence to standard precautions do not address psychosocial aspects or, due to its multifactorial nature, the assessment of adherence is broken down into subscales. This fact led us to include aspects or factors that could affect adherence to standard precautions in a single instrument.

Regarding judges’ classification of items in dimensions, item 41, which addresses garments, was allocated to the organizational dimension, likely mainly because the judges understood this aspect to be linked to institutional protocols. However, item 28, which refers to garment removal that begins inside the room, was allocated to the personal dimension, whereas item 48, which refers to continuity of procedures outside the room, remained in the organizational dimension. Moreover, item 29, which refers to the non-adequate use of PPE due to daily workload, was moved from the organizational to the personal dimension. This demonstrates that content analysis performed based on assessment by judges is a qualitative, subjective approach that is subsequently transformed into a quantitative approach with the CVI use(²⁶). It should be pointed out that this phase precedes psychometric validation, which may end up excluding or reallocating items.

In this first step, only two items were excluded based on the CVI (both related to the personal dimension) for not achieving the established agreement rate of more than 80% among experts. Similar results are described in other studies.

Study limitations

The major limitation of the study was the fact that it was not possible to form a focus group for creating items due to restrictions imposed by the pandemic.

Contributions to nursing

The contribution of this study is the creation of a hospital scale for assessment of the adherence of healthcare providers to good practices for the prevention of the transmission of COVID-19. This study also contributes indirectly to the proposal of interventions that seek the establishment of safe care within the hospital, consequently minimizing the occurrence of morbidity and mortality. The scale will also enable a situational analysis of healthcare providers’ behavior on the frontlines of the battle with COVID-19, resulting in the promotion of educational actions and revision of work processed by administrators, aiming at improving care and reducing infection rates.

CONCLUSIONS

The creation of the scale followed the steps recommended in the literature for the definition of its dimensions and items. Following the assessment by judges, substantial changes were made to the items in the content and face validation step, which occurred in two rounds. This may be attributed to little experience and knowledge of COVID-19 at the time. During the semantic analysis, no further changes to the instrument were made, and the comprehension rate in the target population was adequate. At the end of the process, content and face validity were satisfactory and achieved the parameters recommended in the literature. However, this was a qualitative validation. The instrument may still undergo changes in the psychometric validation step, which is currently underway. Thus, continuity of the study and analysis of the scale’s psychometric properties are recommended.

REFERENCES

• ¹
  Wang C, Horby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. Lancet. 2020;395(10223):470-73. https://doi.org10.1016/S0140-6736(20)30185-9
  » https://doi.org10.1016/S0140-6736(20)30185-9
• ²
  World Health Organization. Interim guidance: Infection prevention and control during health care when COVID-19 is suspected: interim guidance [Internet]. OMS; 19 Mar 2020 [cited 2020 Oct 20]. Available from: https://www.who.int/publications/i/item/10665-331495
  » https://www.who.int/publications/i/item/10665-331495
• ³
  Ferguson NM, Laydon D, Nedjati-Gilani G, Imai N, Ainslie K, Baguelin M, et al. Impact of NonPharmaceutical Interventions (NPIs) to Reduce COVID-19 Mortality and Healthcare Demand. Imperial College COVID-19 Response Team. 2020;1-20. https://doi.org10.25561/77482
  » https://doi.org10.25561/77482
• ⁴
  Li R, Rivers C, Tan Q, Murray MB, Toner E, Lipsitch M. Estimated demand for US hospital inpatient and intensive care unit beds for patients with COVID-19 based on comparisons with Wuhan and Guangzhou, China. JAMA Netw Open. 2020;3(5):e208297. https://doi.org10.1001/jamanetworkopen.2020.8297
  » https://doi.org10.1001/jamanetworkopen.2020.8297
• ⁵
  World Health Organization (WHO). Coronavirus disease (COVID-19) outbreak: rights, roles and responsibilities of health workers, including key considerations for occupational safety and health [Internet]. OMS [cited 2020 Jul 1]. Available from: https://www.who.int/docs/default-source/coronaviruse/who-rights-roles-respon-hw-COVID-19.pdf?sfvrsn=bcabd401_0
  » https://www.who.int/docs/default-source/coronaviruse/who-rights-roles-respon-hw-COVID-19.pdf?sfvrsn=bcabd401_0
• ⁶
  Garner JS. Guideline for isolation precautions in hospitals. The Hospital Infection Control Practices Advisory Committee. Infect Control Hosp Epidemiol. 1996;17(1):53-80. https://doi.org10.1086/647190
  » https://doi.org10.1086/647190
• ⁷
  Centers for Disease Control and Prevention. Interim infection prevention and control recommendations for healthcare personnel during the Coronavirus Disease 2019 (COVID-19) Pandemic [Internet]. 2020 [cited 2020 Jul 1]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/hcp/infection-control-recommendations.html
  » https://www.cdc.gov/coronavirus/2019-ncov/hcp/infection-control-recommendations.html
• ⁸
  Ministério da Saúde (BR). Secretaria de Vigilância em Saúde Programa Nacional de DST e Aids. Manual de adesão ao tratamento para pessoas vivendo com HIV e Aids [Internet]. Brasília (DF): 2008 [cited 2020 Jul 1]. Available fom: http://bvsms.saude.gov.br/bvs/publicacoes/manual_adesao_tratamento_hiv.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/manual_adesao_tratamento_hiv.pdf
• ⁹
  Brevidelli MM. Modelo explicativo da adesão às precauções-padrão: construção e aplicação [Tese]. São Paulo: Escola de Enfermagem, Universidade de São Paulo; 2003.
• ¹⁰
  Brevidelli MM, Cianciarullo TI. Fatores psicossociais e organizacionais na adesão às precauções-padrão. Rev Saúde Pública. 2009;43(6):1-10. https://doi.org10.1590/S0034-89102009005000065
  » https://doi.org10.1590/S0034-89102009005000065
• ¹¹
  Terwee CB, Prinsen CA, de Vet HCW, et al. COSMIN methodology for assessing the content validity of Patient-Reported Outcome Measures (PROMs): user manual [Internet]. 2018[cited 2020 Jul 1]. Available from: www.cosmin.nl
  » www.cosmin.nl
• ¹²
  Sharma SK, Nuttal C, Kalyani V, Hemlata. Clinical nursing care guidance for management of patient with COVID-19. J Pak Med Assoc [Internet]. 2020[cited 2020 Jul 1];70(Suppl.30):S118-S123. https://pubmed.ncbi.nlm.nih.gov/32515397/
  » https://pubmed.ncbi.nlm.nih.gov/32515397/
• ¹³
  Dela Coleta MF. O modelo de crenças em saúde (HBM):uma análise de sua contribuição à psicologia da saúde. Temas Psicol [Internet]. 1999 [cited 2020 Oct 15];7(2):175-82. Available fom: http://pepsic.bvsalud.org/pdf/tp/v7n2/v7n2a07.pdf
  » http://pepsic.bvsalud.org/pdf/tp/v7n2/v7n2a07.pdf
• ¹⁴
  Rosenstock IM. The health belief model: explaining health behavior through expectancies. In: Glanz K, Lewis FM, Rimer BK. Health behavior and health education: theory, research and practice. San Francisco: Jossey-Bass Publishers; 1990. p. 39-62.
• ¹⁵
  Bardin L. Analise de conteúdo. Lisboa: Portugal; Edições 70; 1977.
• ¹⁶
  Pasquali L. Princípios de elaboração de escalas psicológicas. Rev Psiquiatr Clín [Internet]. 1998 [cited 2020 Sep 14];25(5):206-13. Available from: http://mpet.ifam.edu.br/wp-content/uploads/2017/12/Principios-deelaboracao-de-escalas-psicologicas.pdf
  » http://mpet.ifam.edu.br/wp-content/uploads/2017/12/Principios-deelaboracao-de-escalas-psicologicas.pdf
• ¹⁷
  Yusoff MSB. ABC of Content Validation and Content Validity Index Calculation. Educ Med J. 2019.11(2):49-54. https://doi.org10.21315/eimj2019.11.2.6
  » https://doi.org10.21315/eimj2019.11.2.6
• ¹⁸
  Polit DF, Beck CT, Owen SV. Is the CVI an Acceptable Indicator of Content Validity? appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67. https://doi.org10.1002/nur.20199
  » https://doi.org10.1002/nur.20199
• ¹⁹
  Devellis RF. Scale development: theory and applications. 4ª ed. Los Angeles: Sage; 2017. p. 106-51.
• ²⁰
  Polit D, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489-97. https://doi.org10.1002/nur.20147
  » https://doi.org10.1002/nur.20147
• ²¹
  Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the Process of Cross-Cultural Adaptation of Self-Report Measures. Spine. 2000;25(24):3186-91. https://doi.org10.1097/00007632-200012150-00014
  » https://doi.org10.1097/00007632-200012150-00014
• ²²
  Agência Nacional de Vigilância Sanitária (ANVISA). Nota Técnica GVIMS/GGTES/Anvisa No 04/2020. Orientações para serviços de saúde: medidas de prevenção e controle que devem ser adotadas durante a assistência aos casos suspeitos ou confirmados de infecção pelo novo coronavírus (SARS-CoV-2) [Internet]. 31 Mar 2020 [cited 2020 Jul 1]. Available from: https://www20.anvisa.gov.br/segurancadopaciente/index.php/alertas/item/nota-tecnica-n-04-2020-gvims-ggtes-anvisa-atualizada
  » https://www20.anvisa.gov.br/segurancadopaciente/index.php/alertas/item/nota-tecnica-n-04-2020-gvims-ggtes-anvisa-atualizada
• ²³
  Centers for Disease Control and Prevention. Using Personal Protective Equipment (PPE). 2020[Internet]. 11 March 2021[cited 2021 March 11]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/hcp/using-ppe.html
  » https://www.cdc.gov/coronavirus/2019-ncov/hcp/using-ppe.html
• ²⁴
  World Health Organization (WHO). Rational use of personal protective equipment for COVID-19 and considerations during severe shortages [Internet]. WHO. 23 December 2020. [cited 2021 March 11]. Available from: https://www.who.int/publications/i/item/rational-use-of-personal-protective-equipment-for-coronavirus-disease-(COVID-19)-and-considerations-during-severe-shortages
  » https://www.who.int/publications/i/item/rational-use-of-personal-protective-equipment-for-coronavirus-disease-(COVID-19)-and-considerations-during-severe-shortages
• ²⁵
  Organização Panamericana de Saúde. Prevenção e controle de infecção durante os cuidados de saúde quando houver suspeita de infecção pelo novo coronavírus (nCoV): diretrizes provisórias [Internet]. 25 de janeiro 2020[cited 2021 March 11]. Available from: https://iris.paho.org/bitstream/handle/10665.2/51910/9789275722039_por.pdf?sequence=1&isAllowed=y
  » https://iris.paho.org/bitstream/handle/10665.2/51910/9789275722039_por.pdf?sequence=1&isAllowed=y
• ²⁶
  Kimberlin CL, Winterstein AG. Validity and reliability of measurement instruments used in research. Am J Health Syst Pharm. 2008;65(23):2276-84. https://doi.org10.2146/ajhp070364
  » https://doi.org10.2146/ajhp070364

Edited by

EDITOR IN CHIEF: Dulce Barbosa

ASSOCIATE EDITOR: Fátima Helena do Espírito Santo

Publication Dates

• Publication in this collection
  29 July 2022
• Date of issue
  2022

History

• Received
  11 Nov 2020
• Accepted
  24 Apr 2022