Indicators to assess the quality of programs to prevent occupational risk for tuberculosis: are they feasible?

Abstract

Objective:

to analyze the feasibility of quality indicators for evaluation of hospital programs for preventing occupational tuberculosis.

Method:

a descriptive cross-sectional study. We tested indicators for evaluating occupational tuberculosis prevention programs in six hospitals. The criterion to define feasibility was the time spent to calculate the indicators.

Results:

time spent to evaluate the indicators ranged from 2h 52min to 15h11min 24sec. The indicator for structure evaluation required less time; the longest time was spent on process indicators, including the observation of healthcare workers' practices in relation to the use of N95 masks. There was an hindrance to test one of the indicators for tuberculosis outcomes in five situations, due to the lack of use of tuberculin skin test in these facilities. The time requires to calculate indicators in regarding to the outcomes for occupational tuberculosis largely depends upon the level of organizational administrative structure for gathering data.

Conclusions:

indicators to evaluate the structure for occupational tuberculosis prevention are highly feasible. Nevertheless, the feasibility of indicators for process and outcome is limited due to relevant variations in administrative issues at healthcare facilities.

Descriptors:
Tuberculosis; Quality Indicators, Health Care; Occupational Risk

Resumo

Objetivo:

analisar a viabilidade de indicadores de qualidade para avaliação de programas hospitalares de prevenção de tuberculose ocupacional.

Método:

estudo descritivo transversal. Testaram-se indicadores de avaliação de programas de prevenção de tuberculose ocupacional em seis hospitais. O critério para definir a viabilidade foi o tempo necessário para aplicar os indicadores.

Resultados:

o tempo necessário para avaliar os indicadores variou de 02'52'' até 15h11'24''. O indicador para a avaliação da estrutura demandou menor tempo; o maior tempo foi utilizado com os indicadores de processo, incluindo a observação das práticas dos trabalhadores de saúde em relação ao uso de máscaras N95. Um dos indicadores de resultados de tuberculose deixou de ser testado em cinco situações devido à falta de uso do teste tuberculínico nessas instituições. O tempo necessário para aplicar indicadores em relação aos resultados de tuberculose ocupacional depende em grande parte do nível da organização da estrutura administrativa para a coleta de dados.

Conclusões:

os indicadores de avaliação da estrutura de prevenção de tuberculose ocupacional são altamente viáveis. No entanto, a viabilidade de aplicação dos indicadores de processo e de resultado é limitada devido a variações relevantes em questões administrativas nas instituições de saúde.

Descritores:
Tuberculose; Indicadores de Qualidade em Assistência à Saúde; Riscos Ocupacionais

Resumen

Objetivo:

analizar la viabilidad de los indicadores de calidad de la evaluación de los programas hospitalarios para la prevención de la tuberculosis en el trabajo.

Metodología:

estudio descriptivo transversal. Se probaron los indicadores dirigidos a evaluar los programas para la prevención de la tuberculosis laboral en seis hospitales. El criterio para definir la viabilidad fue el tiempo para aplicar los indicadores.

Resultados:

el tiempo empleado para la evaluación de los los indicadores varió desde 02'52 '' hasta 15h11'24 ''. El indicador para la evaluación de la estructura requiere menos tiempo; se invirtió más tiempo en los indicadores de proceso, lo que incluye la observación de las prácticas de los empleados del cuidado de salud en relación con el uso de máscaras N95. No se pudo probar uno de los indicadores de resultados de tuberculosis en cinco situaciones debido a la falta de uso de la prueba de la tuberculina en estas centros. El tiempo necesario para aplicar los indicadores en relación con los resultados por tuberculosis laboral depende en gran medida del nivel de organización de la estructura administrativa para la recopilación de datos.

Conclusiones:

los indicadores para evaluar la estructura para la prevención de la tuberculosis laboral son altamente factibles. Sin embargo, la viabilidad de aplicación de los indicadores de proceso y el resultado es limitada debido a las variaciones relevantes en cuestiones administrativas en los centros sanitarios.

Descriptores:
Tuberculosis; Indicadores de Calidad de la Atención de Salud; Riesgos Laborales

Introduction

Tuberculosis continues to be a threat worldwide. Consequently, many healthcare workers (HCW) are at risk of being infected and acquiring this disease¹. Good prevention programs should be established to avoid this undesirable outcome in healthcare facilities. To evaluate such programs, quality indicators can be used to identify the level of compliance for recommended practices.

Quality indicator technology has been increasingly used for evaluating health care practices. They are quantitative measures of features or attributes of a given process or system², which may indicate the heath care quality provided, as well as specific needs for improvement³. Three classical categories have been used for their classification: structure, process and outcome^2,4.The advantage of one over the other lies in the characteristics of the phenomenon to be measured.

Structure indicators refer to the features required, such as human resources, equipment, information systems, etc. Process indicators measure the dynamics of a given process, or how this particular process was performed. Outcome indicators measure the frequency in which event occurs, and assess final goals, such as mortality, morbidity or patient satisfaction^2,5. Ideal indicators include features such as acceptability, objectivity, effectiveness, reliability, feasibility and availability, communication, interpretability, reproducibility, context, sensitivity to change, efficiency, and comparability⁶.

In 2006, a group of researchers in Brazil constructed and validated a set of indicators designed to evaluate the quality of programs for healthcare-associated infection (HAI) prevention, including occupational tuberculosis. They can also be used to gauge the extent to which the control of HAI differs between different institutions^5,7. Although the content was validated by professional experts, these indicators have not yet been fully tested.

Due to great difficulty in finding patterns for feasibility assessment in the literature, the best criteria for defining feasibility was previously discussed in a focus group with specialists^6,8. The criterion "time" was chosen as a way of classifying these indicators as feasible. Providing the extent of time spent in measuring the indicator is as short as possible, the indicator is considered feasible. The shorter the time, the lower the human resources expense, and the more likely the indicator is to be widely use.

In the present study we aimed to analyze the feasibility of these quality indicators for the evaluation of programs for preventing occupational tuberculosis.

Methods

This descriptive, cross-sectional study tested the feasibility of quality indicators aimed at evaluating elements of structure, process and outcome of occupational tuberculosis prevention programs in different healthcare facilities. Time required for the calculationof the indicators was assessed as a measure of feasibility.

The quality indicators are described in Figure 1, with a brief description, formula, ideal values, sources of information, components of analysis, evaluation criteria and sample.

Figure 1
Indicators for evaluating prevention and control programs for biological occupational risk of tuberculosis, according to Takahashi⁷. São Paulo, SP, Brazil, 2011

Indicators were applied in six different institutions in the city of São Paulo, Brazil, which met the following requirements: a) acute care hospital, b) public or private setting, c) caring for patients with suspected or confirmed pulmonary or laryngeal tuberculosis in the bacillary phase, and d) having a formal Healthcare-associated Infection Control Committee (HICC).

Selected variables were used to characterize the participant institutions and to identify the components that may contribute to the variations in the time required to calculate the indicators. These variables included the number of active beds; the average prevalence of daily (or monthly) inpatients with pulmonary orlaryngeal tuberculosis bacillus and aerosol precautions indicated, number of employees in the institution, and nature of the institution (public / private / philanthropic).

Data were collected using a standardized form, and the time was measured using a chronometer. Secondary variables were collected in order to identify elements justifying the time spent to collect each indicator. Periods of interruption and time intervals were deducted from the overall time span of the activity. Also, the time spent on healthcare facility characterization, as well as the time required to access the hospital facilities (reception, elevators, etc.) was not included in data collection. Indicators were tested by the same researcher in all healthcare facilities (T.R.S.).

The variables of analysis were the time spent on: a) data collection, and b) data consolidation and analysis. We compared the time spent on testing of each indicator in the different institutions.

Data were collected from December 2010 to July 2012. This period was required to complete data collection in all six institutions, due to the small number of inpatients having pulmonary or laryngeal tuberculosis who were placed in aerosol precautions in some of the hospitals.The data analysis was descriptive.

Results

Among the six institutions in which the quality indicators were tested, four were general hospitals, one was a hospital specializing in infectious disease, and one was a general hospital, although it served as a reference site for tuberculosis treatment (Table 1). Among 2,655 beds in six institutions, 55.54% (1,480) were public. Altogether, these institutions had approximately 24,271 health workers, 45.91% (11,145) in the public sector. Among the 690 hospitalized patients with diagnosed or suspected pulmonary or laryngeal tuberculosis, 94.63% (653) were admittedtopublic institutions.

Table 1
Characteristics of the institutions surveyed and time spent for collection and consolidation of quality indicators for occupational tuberculosis prevention programs. São Paulo, SP, Brazil, 2011-2012

Institution
Characteristics	A	B	C	D	E	F
Number of active beds Number of patients with laryngeal or pulmonary tuberculosis bacillus, hospitalized in the last year, indicating isolation.	220	983	341	220	614	277
	489	96	11	11	15	68
Number of health workers in the institution	1737	6000	4126	1300	9700	1408
Natureoffunding	Public	Public	Private	Private	Private	Public
Time required for data collection and consolidation of indicator (h/min/sec)
TORPS*	00:24:39	00:18:38	00:17:56	00:12:19	00:23:54	00:18:25
TOI† TOSCI‡	01:02:03	00:19:40	-	00:04:15	00:02:52	00:03:27
	-	00:06:58	-	-	-	-
TOPWC§	04:44:28	04:56:15	14:23:13	15:11:24	08:37:49	11:44:58
Data consolidation	01:17:57	01:13:25	01:00:07	01:04:30	00:58:39	01:01:08
Total time	07:29:08	06:55:33	15:51:16	16:32:28	10:03:14	13:17:58

• ^*
  Tuberculosis Occupational Risk Prevention Program Structure.^† Tuberculosis Occupational Incidence.^‡ Tuberculosis Occupational Skin Conversion Incidence.^§Tuberculosis Occupational Prevention Workers Compliance.

All evaluated facilities had the same recommendation for the use of a N95 particulate respirator: to put it on in the anteroom or in the hallway before entering the room of a patient with known or suspected pulmonary or laryngeal tuberculosis bacillus.

The TORPS indicator resulted in minimal effort and time spent on its application in all institutions (Table 1) Regarding the TOSCI indicator, the information necessary for its calculation was not found in five of the six health institutions. Several arguments were used to report the absence of the use of Tuberculosis Skin Test (TST): lack of trained personnel to perform the test, the porosity of collection, frequent lack of HCW follow-up for appropriate characterization of the reaction; difficulty in identifying the exact time period of the HCW's exposure to the mycobacterium.

The TOI indicator was collected in five of the six institutions; there was only one healthcare facility in which data were not organized in such a way that it could be collected. The time used to collect this indicator was not toolong, but depended on the level of data organization.

The TOPWC indicator required greater time for calculation(Table 1). To note, public hospitals were more likely to require less time to collect data than the private sector. In public hospitals there are usually more patients admitted with tuberculosis, therefore it was possible to observe two or more patients simultaneously, thus reaching 51 observations more rapidly.

Discussion

Many quality indicators have been proposed in the literature, however few have been evaluated regarding their feasibility for application, which creates a gap between theory and practice. Nevertheless, the recommendation for their use is quite frequent. To our knowledge, the present study is the first to evaluate the feasibility of quality indicators, using as the criterion the time spent on administering / calculatingthem.

Information on quality of care depends upon data availability. Therefore, quality is difficult to measure without correct and consistent information, which is often unavailable⁸. A previous study evaluated the feasibility of quality indicators related to radical prostatectomy and concluded that indicators not obtaining more than 25.9% of the necessary information were considered unenforceable⁹. It has also been previously shown that quality indicators for antibiotic treatment of complicated urinary tract infections were considered feasible if the data necessary to score the indicator can be abstracted from the available data for >70% of cases¹⁰. Indicators should require ease of obtaining data or ease of availability of the data as a condition of feasibility, resulting in minimal effort and additional cost^6,11. Because time spent on data gathering and analysis reflects both on efforts and cost, less time means higherf easibility.

Although time spent on the application of quality indicators of an occupational tuberculosis prevention program may vary in different healthcare facilities, some common features were noted from this study. For instance, the indicator that evaluated the structure of the program (TORPS) proved to be highly feasible. This indicator has characteristics suggestive of being used for external audits and evaluations. On the other hand, the process indicator (TOPWC) requires greater dedication of professional time for its application. This indicator should be used preferentially by healthcare facilities that have a higher number of in-patients requiring special precautions for tuberculosis, aiming to evaluate compliance with the use of the N95 mask by HCWs. As a suggestion, TOPWC could be applied biannually, or after major intervention and training programs.

It is a matter for discussion as to why, despite recommendations, some healthcare facilities in Brazil are not using the TST routinely, as we demonstrated in our sample. As an outcome to be measured, it was shown that the indicator for skin conversion (TOSCI) was not feasible due to this lack of compliance. The Centers for Disease Control recommends the use of the TST whenever there is the possibility of high exposure to tuberculosis¹². HCWs should be periodically screened for latent tuberculosis infection using TST. As pointed out, concerning the healthcare facilities, many operational issues can interfere in the process. Among these issues, are the high turnover of HCWs, the limitations of the TST interpretation, and a potential booster effect of the BCG vaccine^13-15.In order to overcome the booster effect, a two-step TST has been suggested in the literature^15-17.The TST has a high sensitivity, but lacks specificity in a vaccinated population, such as the HCWs in Brazil. Due to this feature, countries such as France and Japan are now recommending, with some restrictions, the gama-interferon release assays as a substitute for TST^18-19. To note, in our sample, none of the healthcare facilities that were not using TST provided any other screening measure as a substitute.

The main outcome indicator (TOI), which measures the incidence of cases of tuberculosis among HCW, is quite simple to obtain, provided the Occupational Medicine Service has a structured form to record such cases. Usually cases of occupational tuberculosis are not as frequent as to warrant a great deal of effort in recording them. Besides this, the number of exposed HCWs is, in general, quite steady and does not require a sophisticated system to collect the information. Despite this, many healthcare facilities are not aware of monitoring the annual incidence of occupational cases of tuberculosis.

The World Health Organization (WHO) shows that tuberculosis mortality in Brazil in 2013 was 3.2/100.000 and the prevalence was 57/100.000²⁰. Some authors have published similar results. A Peruvian study found a tuberculin test conversion incidence in medical students of approximately 3%²¹. A Brazilian study conducted in Belo Horizonte, MG, Brazil, where the tuberculosis incidence rate is 23/100.000, had the cooperation of 251 HCWs. The TST conversion was 5.1%, with the risk of infection of 1.4²². A study aimed to identify the TST conversion rate of HCWs with previously negative TST results who had been working for less than 1 year in a hospital in Botswana, where tuberculosis is highly endemic. This population had a conversion rate of 4.2% for the entire group studied, or 6.87 per 1000 person-weeks²³.

A Chinese study showed that the health care workers' annual tuberculosis notification rates were lower than the general population. Healthcare workers with tuberculosis were a mean of 35.5 years old, with females out numbering males (58.0%>42.0%). The proportion of pulmonary tuberculosis was significantly higher among the women compared with men (88.5%>83.4%, p = 0.031). This study suggested that the priority for tuberculosis prevention in healthcare institutions should be given to the young female HCWs¹.

An Argentinean study that included 15,276 HCWs from 15 centers found a mean incidence rate of tuberculosis in 111.3/100,000 HCWs²⁴; A Brazilian study demonstrated incidence rates in the general population of approximately 62/100,000, a prevalence of tuberculosis infection in HCW of 63.1% and an annual rate of tuberculin conversion of 10.7%²⁵. In such an epidemiologic context, monitoring the incidence of occupational tuberculosis and the TST conversion can aid institutions in planning and evaluating strategies for occupational tuberculosis prevention, as demonstrated by other authors^13,15.

With 1.5 million deaths in 2013 and 5.7 million new cases of tuberculosis disease, the WHO goal is to dramatically reduce the global burden of tuberculosis by 2015²⁰. For this control, it will be necessary to include the successful development and application of new drugs, diagnostics, vaccines, and prevention tools as well as a clearer understanding of the impact of social and economic determinants of this disease in the health sector. The quality indicators of programs for prevention of occupational tuberculosis evaluated in the present study were shown to be feasible. Since HCWs have 2- 50 times the chance of acquiring the disease than people in the general population, these indicators can help institutions prevent occupational tuberculosis. Therefore, we recommend their application at least once a year in healthcare facilities that frequently deal with patients affected by tuberculosis.

The results are limited by the small amount of participant institutions, which only enables a suggestion of possible relationships between indicators and the institutional profile. Further studies should include multiple institutions to enable the investigation of relationships between the nature of the institution and the feasibility of applying the quality indicators. , There were not many objective criteria found in the literature that allowed for the evaluation of the applicability of indicators, so it was decided to use time as a marker. However, we understand that this is a specific perspective that limits the study.

This study brings new insight to the applicability of previously validated quality indicators, revealing that even a validated indicator may not have all the properties of applicability; this approach needs to be considered to suggest recommendations for their use.

Moreover, strengths in the structure assessment, and weaknesses in the process and outcomes assessments, have been identified. Areas to be improved include maintaining periodic screening for latent tuberculosis using TST, monitoring the annual incidence of occupational cases of tuberculosis, and evaluating compliance with occupational prevention.

Conclusion

The indicators to evaluate the structure for occupational tuberculosis prevention are highly feasible. The feasibility of applying indicators for process and outcome is limited, due to relevant differences in administrative issues at healthcare facilities, such as the system for data archiving and management.

Acknowledgments

To Esperança Abreu Santos, Vera Lúcia Borrasca, Cristiane Schmitt, Anna Sara Levin, Denilson de Oliveira Reis, Ícaro Boszczowski, Julia Yaeko Kawagoe and Priscila Gonçalves for their cooperation in acting as the contact reference,during data collection, in their institutions.

References

• ¹
  Bin C, Xiaomeng W, Jiemin Z, Songhua C, Beibei W, Hui-Chi Yeh, et al. Tuberculosis among Healthcare Workers in Southeastern China: A Retrospective Study of 7-Year Surveillance Data. Int J Environ Res Public Health. 2014;11:12042-52.
• ²
  Silva CPR, Lacerda RA. Validation of a proposal for evaluating hospital infection control programs. Rev Saúde Pública. 2011;45(1):121-8.
• ³
  Fröhlich SA, Mengue SS. Are the drug prescription quality indicators of the World Health Organization still valid? Ciênc Saúde Coletiva. 2011;16(4):2289-96.
• ⁴
  Moore L, Lavoie A, Bourgeois G, Lapointe J. Donabedian's structure-process-outcome quality of care model: Validation in an integrated trauma system. J Trauma Acute Care Surg. 2015;78(6):1168-75.
• ⁵
  Schmitt C, Lacerda RA, Padoveze MC, Turrini RN. Applying validated quality indicators to surgical antibiotic prophylaxis in a Brazilian hospital: learning what should be learned. Am J Infect Control. 2012;40(10):960-2.
• ⁶
  Peña A, Virk SS, Shewchuk RM, Allison JJ, Williams OD, Kiefe CI. Validity versus feasibility for quality of care indicators: expert panel results from MI-Plus study. International J Qual Health Care. 2010;22(3):201-9.
• ⁷
  Takahashi RF, Gryschek AL, Izumi Nichiata LY, Lacerda RA, Ciosak SI, Gir E, et al. Evaluation of biologic occupational risk control practices: quality indicators development and validation. Am J Infect Control. 2010;38(4):e16-20.
• ⁸
  Ferrua M, Couralet M, Nitenberg G, Morlin S, Serin D, Minvielle E. Development and feasibility of a set of quality indicators relative to the timeliness and organisation of care for new breast cancer patients undergoing surgery. BMC. 2012;12:167. doi: 10.1186/1472-6963-12-167.
  » https://doi.org/10.1186/1472-6963-12-167
• ⁹
  Chan EO, Groome PA, Siemens DR. Validation of quality indicators for radical prostatectomy. Int J Cancer. 2008;123(11):2651-7.
• ¹⁰
  Hermanides HS, Hulscher JL, Schouten JA, Prins JM, Geerlings. Development of quality indicators for antibiotic treatmente of conplicated urinary trat infections: a first step to measure and improve care. Clin Infect Dis.2008;46:703-11.
• ¹¹
  Dy SM, Lorenz KA, O´Neill SM, Asch SM, Walling AM, Tisnado D, et al. Cancer Quality-ASSIST supportive oncology quality indicator set: feasibility, reliability, and validity testing. Cancer.2010;116:3267-75.
• ¹²
  Centers for Disease Control and Prevention. Guidelines for Preventing the Transmission of Mycobacterium tuberculosis in Health-Care Settings, 2005. MMWR. 2005;54(17):3-42.
• ¹³
  Casas I, Esteve M, Guerola R, García-Olivé I, Roldán-Merino J, Martinez-Rivera C, et al. Incidence of tuberculosis infection among healthcare workers: Risk factors and 20-year evolution. Respir Med. 2013;107(4):601-7.
• ¹⁴
  Hirama T, Hagiwara K, Kanazawa M. Tuberculosis screening programme using the QuantiFERON-TB Gold test and chest computed tomography for healthcare workers accidentally exposed to patients with tuberculosis. J Hosp Infect. 2011;77:257-62.
• ¹⁵
  Muzzi A, Seminari E, Feletti T, Scudeller L, Marone P, Tinelli C, et al. Post-exposure rate of tuberculosis infection among health care workers measured with tuberculin skin test conversion after unprotected exposure to patients with pulmonary tuberculosis: 6-year experience in an Italian teaching hospital. BMC Infect Dis. 2014;14:324. doi: 10.1186/1471-2334-14-324.
  » https://doi.org/10.1186/1471-2334-14-324
• ¹⁶
  Oliveira AMVL, Ruffino-Netto A, Paniago AMM, Oliveira OA, Marques M, Cunha RV, et al. Tuberculin skin test: operational research in the state of Mato Grosso do Sul, Brazil. J Bras Pneumol. 2011;37(5):646-54.
• ¹⁷
  Guimarães Teixeira E, Kritski A, Ruffino-Netto A, Steffen R, Lapa e Silva JR, Beloa M, et al. Medical students at risk of nosocomial tuberculosis. J Hosp Infect. 2011;77:76-9.
• ¹⁸
  Rubbo PA, Nagot N, Moing VL, Brabet M, Bourdin A, Nogué E, et al. Multicytokine detection improves latent tuberculosis diagnosis in health care workers. J Clin Microbiol. 2012;50(5):1711-7.
• ¹⁹
  Uto T, Yasuda K, Sagisaka S, Sato J, Imokawa S, Uemura N, et al. Serial QuantiFERON TB-2G Testing Over a Four-year Period in Healthcare Workers at a City Hospital. Intern Med. 2014;53:1119-24.
• ²⁰
  World Health Organization (WHO). Global tuberculosis report 2014. Geneva, Switzerland: WHO; 2014.
• ²¹
  Delgado KC, Bravo SG, Huamán LN, Portella RQ, Montag AR, Alejos AR, et al. Estudiantes de medicina en riesgo: prevalencia e incidencia de conversión de PPD. Rev Chilena Infectol. 2012;29(4):375-81.
• ²²
  Miranda SS , Oliveira AC, Santos AX, Prado DP, Soares CL, Nery RS, et al. Positive tuberculin test and risk of infection by Mycobacterium tuberculosis in a tuberculosis clinic settled in an upright building, in Minas Gerais, Brazi. Rev Méd. Chilena. 2012;140(8):1022-7.
• ²³
  Szep Z, Kim R, Ratcliffe SJ, Gluckman S. Tuberculin skin test conversion rate among short-term health care workers returning from Gaborone, Botswana. Travel Med Infect Dis. 2014;12(4):396-400.
• ²⁴
  Gonzales C, Araujo C, Agoglia R, Hernandez S, Seguel I, Saenz C. Tuberculosis en trabajadores de la salud. Medicina. 2010;70:23-30.
• ²⁵
  Roth VR, Garret DO, Laserson KF, Starling CE, Kritski AL, Medeiros EA, et al. A multicenter evaluation of tuberculin skin test positivity among health care workers in Brazilian hospitals. Int J Tuberc Lung Dis. 2005;9:1335-42.

Publication Dates

• Publication in this collection
  2016

History

• Received
  21 Mar 2015
• Accepted
  07 Aug 2015