Training program on microbiological test collection material methods at a teaching hospital: investment and result assessment

Programa de capacitación sobre el método de colecta de material para análisis microbiológico en un hospital-escuela: inversión y evaluación de los resultados

Abstracts

Este estudo teve por objetivo avaliar os resultados, os custos diretos e o investimento de um programa de treinamento sobre coleta de materiais para exame microbiológico em um hospital de ensino. Foram consideradas como medida de resultados as coletas de exames que não seguiram os critérios estabelecidos (anomalia). O sistema de custeio variável e por absorção foram utilizados, respectivamente, para cálculo dos custos diretos e investimento. Dos 11.893 materiais colhidos foram evidenciadas anomalias em 59 (0,5%). O custo direto foi de R$ 154,10, e o investimento no treinamento, de R$ 2.431,29. Os achados permitiram concluir que o número de anomalias evidenciadas representou um baixo percentual em relação ao total de material colhido para exames microbiológicos, não se constituindo em ponto crítico que justificasse a continuidade do treinamento e conseqüente investimento.

custos e análise de custo; educação; enfermagem; microbiologia; técnicas e procedimentos de laboratório

ORIGINAL ARTICLE

Training program on microbiological test collection material methods at a teaching hospital: investment and result assessment

Programa de capacitación sobre el método de colecta de material para análisis microbiológico en un hospital-escuela: inversión y evaluación de los resultados

Marli de Carvalho Jericó^I; Valéria Castilho^II; Márcia Galan Perroca^III

^IRN, Doctoral student in Nursing, Faculty, FAMERP, São José do Rio Preto

^IIRN, PhD in Nursing, Faculty, University of São Paulo at College of Nursing

^IIIRN, PhD in Nursing, Faculty, FAMERP, São José do Rio Preto

ABSTRACT

This study aimed at evaluating the results, direct costs and investment of a training program on microbiological test material collection at a teaching hospital. Test collections that did not follow the established criteria (failure) were considered as the result measure. Variable and absorption costing were used to calculate direct costs and investments, respectively. Of the 11,893 collected materials, failures were evidenced in 59 (0.5%). Direct cost corresponded to R$ 154.10 and R$ 2,431.29 was invested in training. These findings revealed that the evidenced number of anomalies (failures) represented a low percentage in relation to the total collected material for microbiological exams. Therefore, this should not be considered a critical point that justifies the continuity of the training and, consequently, the investment.

Descriptors: costs and cost analysis; education; nursing; microbiology; laboratory techniques and procedures

RESUMEN

Este estudio tiene como objetivo evaluar los resultados, los costos directos y la inversión en un programa de capacitación para la colecta de muestras para análisis microbiológico en un hospital-escuela. Fueron consideradas como medida de resultados las colectas de análisis que no siguieron los criterios establecidos (anomalía). Los sistemas de costo variable y por absorción fueron utilizados, respectivamente, para calcular los costos directos y la inversión. De las 11,893 muestras recogidas, se manifestaron anomalías en 59 (0,5%). El costo directo fue de US$ 84.67, y la inversión total en el programa de US$ 1,335.87. Los resultados permitieron concluir que el número de anomalías evidenciadas significaba un bajo porcentaje con respecto al total de material recogido para análisis microbiológico y no constituía un punto crítico que justificase la continuación del capacitación y la consecuente inversión.

Descriptores: costos y análisis de costo; educación; enfermería; microbiología; técnicas y procedimientos de laboratorio

INTRODUCTION

The cost of quality has been defined as any service expense exceeding planned levels, provided that the activity has been carried out correctly since the beginning⁽¹⁾. It can be classified as voluntary (prevention and assessment cost) and involuntary cost, resulting from internal and external errors. Internal errors occur before the service is transferred to the client and are associated with rework expenses, avoidable losses in the process and others⁽²⁾.

In this study, we aimed to identify internal failures in the microbiological test material collection method after the implementation of a training program by the Continuing Education Center (CEC) of a health care institution. These failures were called anomalies, according to total quality program terminology. The term anomaly can be defined as deviations from normal and expected conditions for the functioning of a standardized process⁽³⁾.

In hospital organizations, maintaining permanent training programs is important, as preparing human resources is the best way to improve care quality and control or reduce hospital costs⁽⁴⁾. Holding training programs that achieve a maximum result level at minimum expenses started to be a challenge for CEC⁽⁵⁾ nurses. In other words, any investments made are expected to generate return.

Thus, it becomes essential to assess organizational processes and programs⁽⁶⁾. There are four assessment stages: participants' reaction, learning, behavior and organizational results⁽⁷⁾. A fifth level - Return on Investment (ROI) can also be added⁽⁸⁾. While some researchers⁽⁷⁾ only identify program benefits, others⁽⁸⁾ convert these benefits into monetary values and compare them with the program's total cost.

In international literature, economic assessment of nursing training programs has been emphasized through cost-benefit and cost-efficacy analyses. A study on venipuncture training⁽⁹⁾ in the United Kingdom identified the cost of resources and their impact on the nurses' daily routine. Another study at an American institution compared two training cost assessment methods⁽¹⁰⁾. Yet another research developed a proportion formula to calculate the cost-efficacy of staff training programs. This formula takes into account cost/participant/hour, cost of learning acquisition, cost of learning application, cost of reinforcement and additional costs⁽⁵⁾.

In continuing nursing education, nurse managers are progressively leaving behind concerns about the quantitative aspect of training and its immediate impact. Instead, they become aware of the need to actually measure the results and assess the costs of these programs.

When nurses are called upon to participate in a cost reduction policy, their acquired knowledge can actively contribute to a more effective control of resources at their work unit, instead of merely serving as a depository of administrative information. Thus, they can propose measures to avoid resource waste and rework by the nursing team⁽¹¹⁾.

OBJECTIVES

- Identify the incidence of anomalies in microbiological test material collection at a teaching hospital;

- Examine the direct costs of laboratory material and labor in the inadequate collection of this material;

- Assess total investments in the training program.

MATERIAL AND METHOD

During 1999, the Continuing Education Center (CEC) of a large teaching hospital in São José do Rio Preto organized a training program about microbiological test material collection, aimed at all nursing team members.

To develop this descriptive and exploratory study, we considered microbiological test material collection techniques that did not comply with the established criteria and orientations by the CEC training program (anomaly).

Data were collected through anomaly notifications the microbiology laboratory sent to the CEC between June and December 1999, after obtaining authorization from the Research Ethics Committee at the place of study.

The investigated anomalies were catheter tip, sputum, surgical wound secretion, feces, urine and blood cultures. A list of material to be used for each procedure was obtained from the CEC, as well as all collected material from the Microbiology Cost Center. The hospital purchase sector provided the unit cost for each material.

In order to calculate anomaly cost, we used the variable or direct costing system, which distributes all variable (direct and indirect) costs. Indirect fixed cost is treated as an expense directly in the result⁽¹²⁾. This system is recommended for the management area because it identifies actually consumed resources in service production, which is very useful for decision making.

Labor cost was verified in view of the activities performed by the nursing auxiliary who collects the material, and the operational auxiliary responsible for transporting material from the laboratory to the unit. Calculations considered the base salary for each category as well as the mean time needed for collection and transport activities, estimated at 15 minutes.

Investments in the training program were calculated through the absorption costing method, characterized by the appropriation of all production, fixed, variable, direct and indirect costs, which are equally distributed among services, while indirect costs were allocated⁽¹³⁾. To assess total investments in the training program, we used the sum of direct (total training cost) and indirect investments (human resource time made available for learning at the CEC times each category's salary per work hour)⁽¹⁴⁾. The real (R$) and the average dollar rate for 1999 (R$1.82) were used in all calculations.

RESULTS AND DISCUSSION

We will present and discuss the results according to the proposed study objectives, that is, incidence of anomalies, places of occurrence, direct cost of samples for laboratory test collection and investments in the training program.

Data in Table 1 show that 11,893 materials were collected and notified to the CEC. Anomalies were identified in 59 of these (0.5%). This figure reveals technical quality in the collection of this material and may represent the impact of the training program on trained professionals' performance, as no earlier assessment was available to compare the obtained results.

Catheter tip collections were responsible for 28 (47.5%) anomalies, which mainly occurred in June, when eight (7.4%) cases were found. The highest anomaly incidence level was found in September, with 11 (10.9%) cases, in comparison with the other months under study.

The following were characterized as anomalies in microbiological test material collection: Catheter tip larger than 5 cm and/or immersed in physiological solution; sputum in non-recommended recipient and/or not collected from first sample and/or seal violation; use of inadequate recipient for feces culture collection; urine in non-sterilized container; insufficient quantity and/or inadequate conditioning of surgical wound secretion collection.

Incorrect material collection entails implications for clients (discomfort, risk, longer hospitalization period, inadequate diagnosis and treatment); for the nursing team and laboratory (longer time, rework) and for the institution (unnecessary use of material and higher costs). Therefore, the identification and implementation of actions to correct anomalies should be a constant and systemized process⁽³⁾ in nurses' clinical and management practice, as well as in the coordination of integrated multidisciplinary team actions.

In combination with training, the definition of methods and processes together with the work team have been indicated as determinant factors to reduce the number of human errors (anomalies)⁽¹⁴⁾. Other very important measures are the uniform realization of technical procedures and making involved professionals aware of how undesirable effects (anomalies) can negatively interfere in the achievement of reliable results⁽¹⁵⁾.

According to Table 2, during the study period, the greater part of the anomalies occurred in non-critical care units A (n = 10) and G (n=5), and in the critical care units F (n = 8), D and H (n=5). Non-critical care units (general and specialty medical and surgical units) were responsible for 45.7% of anomalies, while critical care units (intensive care units- ICUs, operation room, emergency ward) corresponded to 54.3%. Using a statistical test to compare percentages, we found variations from 41 to 67% (p-value = 0.6). In other words, these findings were not statistically significant.

Table 3 shows that, in the total amount of direct costs of microbiological tests with anomalies during the study period ($ 84.58), labor was the highest cost, mainly nursing auxiliaries, which represented $ 40.19 (47.5%). Material were responsible for the smallest part of total costs, i.e. $ 20.96 (24.8%). The highest operational costs for material collection were found in September $ 15.60 (18.4%) and June $ 15.48 (18.3%). The lowest operational cost occurred in December $ 4.38 (5.2%).

A study to assess hygiene and nutrition education applied to kindergarten employees and parents revealed a different material and labor cost distribution. In that research, material represented 87.5% of total costs, against only 12.5% for labor cost⁽¹⁶⁾.

The training program attended 171 participants (monthly average of 24 per training). Initially, the CEC aimed to train the entire nursing team during the year, with a monthly average of 72 participants. Table 4 shows that, in total (sum of direct and indirect investments), $1,335.92 was invested in training about microbiological test material collection. Direct investment (total training cost) corresponded to $ 518.11, against $ 817.71 for indirect investments (human resource time made available for learning at the CEC times each category's salary per work hour). Average per capita cost was $ 8.79.

A study of three training cases (cannulation practice, venipuncture and intravenous drug administration) in the United Kingdom⁽⁹⁾ found a per capita cost of £ 915. A comparative study of two training methods in the United States⁽¹⁰⁾ identified per capita costs of US$7.33 (unit-based method) and US$5.64 (all-day method). Hygiene and nutrition education of parents and kindergarten employees in Brazil showed an average cost of R$ 24.54 per employee⁽¹⁶⁾. However, a study of training programs at a Brazilian hospital institution^(17-18) found a per capita investment of US$ 27.41, that is, 9.2 times lower than the global reference value of US$ 252.

Organizations commonly face difficulties to identify training needs (what should be trained, target public and reason for investment), as well as to define training objectives. Specialized literature also report that many training programs' lack of success is mainly due to the lack of adequate needs assessment⁽¹⁹⁾.

Considering that training needs assessment supports planning, the CEC nurse manager is responsible for an objective diagnosis, based on data collection and not only on nurses' statements. In this study, the lack of a survey before the implementation of the training program, to be compared with posterior data, created doubts about the pertinence of this training.

Since anomalies were more frequent at non-critical care units A and G and at critical care units F and G, the CEC manager's decisions should focus on the nursing team in these units, as well as on catheter tip collection and sputum culture, which were the predominant anomalies.

A low number of anomalies was found (n=59 - 0.5%) in comparison with the total number of materials collected for microbiological testing (n= 11,893). Hence, this is not a critical point that would justify the continuity of training and, consequently, investment.

An analysis of public health service network training⁽²⁰⁾ showed that training has been implemented in a centralized form at these services, without any connection with a strategic staff development and training plan at these institutions. The same study highlights the non-observation of participant inclusion criteria, which constitutes a waste of resources and makes it difficult for participants to transfer the produced knowledge to their activities.

CONCLUSIONS AND RECOMMENDATIONS

In this study, the identification of anomalies was used to measure training results. Anomalies create costs (repair, time, material, etc.) and do not add value, representing losses to the institution. The understanding of costs as a management tool for CEC nurses provides information for operational activity planning, benefits resource allocation and guides relevant investments.

Considering that health institutions usually do not have a formalized investment policy, we recommend that CEC nurses: align programs with the institution's strategic needs; establish participant inclusion criteria based on training needs assessment; design contents with problem-based techniques; measure the obtained results, establishing target levels for improvements in anomaly rates; and add cost studies to the assessment of their programs.

Recebido em: 2.2.2005

Aprovado em: 14.7.2006

• 1. Garvin DA. Gerenciando a Qualidade: a visão estratégica e competitiva. Rio de Janeiro: Qualitymark; 1997.
• 2. Sakurai M. Gerenciamento Integrado de Custos. São Paulo: Atlas; 1997.
• 3. Nogueira LCL. Gerenciando pela qualidade total em saúde. Belo Horizonte: Editora de Desenvolvimento Gerencial; 1999.
• 4.Almeida M H de. Custos hospitalares em enfermagem. Rio de Janeiro: Cultura Médica; 1984.
• 5. Del Bueno DJ, Kelly KJ. How cost-effective is your staff development program? J Nurs Admin 1980 April; 10:31-6.
• 6. Boog GG. Critérios de decisão para investimentos para treinamento e desenvolvimento de recursos humanos. [dissertação]. São Paulo (SP): Escola de Administração de Empresas de São Paulo/FGV;1978.
• 7. Kirkpatrick DL. Evaluating training programs: the four levels. San Francisco: Berret-Koehler Publihers; 1994.
• 8. Phillips JJ. How much is the training worth? Training Develop. 1996 April; 50(4):20-4.
• 9.Nielsen L. A cost benefit analysis of training nurses for extended roles. Nurs Times 2003; 99(28):34-7.
• 10.Bethel PL. Inservice Education:calculating the cost. Continuing Educ Nurs 1992 July-August; 23(4):184-7.
• 11.Padilha MICS. A qualidade da assistência de enfermagem e os custos hospitalares. Rev Hosp Admin Saúde. 1990 julho-setembro; 14(3):128-33.
• 12. Ching HY. Manual de custos de instituições de saúde: sistemas tradicionais de custos e sistema de custeio baseado em atividades (ABC). São Paulo: Atlas; 2001.
• 13. Beulke R. Estrutura e análise de custos. São Paulo: Saraiva; 2001.
• 14.Souza V de, Moura FL, Flores ML. Fatores determinantes e conseqüências de falhas registradas na assistência de enfermagem - um processo educativo. Rev Min Enfermagem. 2002 janeiro-dezembro; 6 (1/2):30-4.
• 15. Gir E, Stuchi RAG, Macedo RFC de, Dresler DE. Ações de enfermagem em hemocultura. Rev Gauch Enfermagem 1998 julho; 19(2):95-105.
• 16. Nogueira LCP. Avaliação do custo-eficácia de uma ação educativa em nutrição e higiene. [dissertação]. São Paulo (SP): Faculdade de Economia e Administração da Universidade de São Paulo/FEA;1999.
• 17. Jericó MC. Análise dos custos dos programas de treinamento e desenvolvimento de pessoal de uma organização hospitalar.[dissertação]. São Paulo (SP): Escola de Enfermagem/USP; 2001.
• 18. Jericó MC, Castilho V. O investimento financeiro em recursos humanos: uma experiência em hospital. Rev Latino-am Enfermagem. 2004 setembro-outubro; 12(5):707-12.
• 19. Magalhães ML, Borges-Andrade JE. Auto e hetero-avaliação no diagnóstico de necessidades de treinamento. Estud Psicol (Natal). [periódico na internet] 2001 janeiro-junho [citado 20 de novembro 2003]; 6(1):33-50.[17telas] Disponível em http://www.scielo.br/scielo.php? script=sci_arttext&pid= S1413294X2001000100005& Ing=pt&nrm=iso
• 20. Melo C, Fagundes N. Discutindo a avaliação de um programa de capacitação para enfermeiros. Rev Bras Enfermagem 1999 janeiro-março; 52(1):37-42.

Publication Dates

• Publication in this collection
  14 Nov 2006
• Date of issue
  Oct 2006

History

• Accepted
  14 July 2006
• Received
  02 Feb 2005