Evaluation of the implementation of a quality system in a basic research laboratory: viability and impacts

Abstracts

OBJETIVO: Avaliar o processo de implantação de um sistema de qualidade em um laboratório de pesquisa básica, avaliando a viabilidade e os impactos dessa melhoria. MÉTODOS: Tratou-se de um estudo qualitativo prospectivo. Utilizou-se a norma NIT DICLA 035 - Princípios das Boas Práticas de Laboratório (BPL) e documentos da Organisation for Economic Co-operation and Development para complementar o planejamento e a implantação de um Sistema de Gestão da Qualidade, em um laboratório de pesquisa básica. Em paralelo, utilizou-se a ferramenta PDCA para definir os objetivos de cada etapa de implantação do sistema de qualidade. RESULTADOS: Este trabalho possibilitou ao laboratório atender requisitos solicitados pela norma NT DICLA 035 e implementá-los durante a execução de um projeto, dentre eles a elaboração de documentos, bem como estabelecer rotinas importantes para o andamento do mesmo, como a identificação de não conformidades, rastreabilidade de dados e calibração de equipamentos. CONCLUSÃO: A implantação do Sistema da Qualidade BPL, nesse cenário, é viável, gerando impactos positivos na rotina do laboratório.

Laboratórios; Pesquisa básica; Gestão de qualidade; Certificação

HEALTH ECONOMICS AND MANAGEMENT

Evaluation of the implementation of a quality system in a basic research laboratory: viability and impacts

Hilda Carolina de Jesus Rios Fraga^I; Kiyoshi Ferreira Fukutani^II; Fabiana Santana Celes^II; Aldina Maria Prado Barral^II; Camila Indiani de Oliveira^II

^IUniversidade Federal da Bahia - UFBA, Salvador (BA), Brazil

^IIFundação Oswaldo Cruz - FIOCRUZ, Salvador (BA), Brazil

^{Corresponding author}

ABSTRACT

OBJECTIVE: To evaluate the process of implementing a quality management system in a basic research laboratory of a public institution, particularly considering the feasibility and impacts of this improvement.

METHODS: This was a prospective and qualitative study. We employed the norm "NIT DICLA 035 - Princípios das Boas Práticas de Laboratório (BPL)" and auxiliary documents of Organisation for Economic Co-operation and Development to complement the planning and implementation of a Quality System, in a basic research laboratory. In parallel, we used the PDCA tool to define the goals of each phase of the implementation process.

RESULTS: This study enabled the laboratory to comply with the NIT DICLA 035 norm and to implement this norm during execution of a research study. Accordingly, documents were prepared and routines were established such as the registration of non-conformities, traceability of research data and equipment calibration.

CONCLUSION: The implementation of a quality system, the setting of a laboratory focused on basic research is feasible once certain structural changes are made. Importantly, impacts were noticed during the process, which could be related to several improvements in the laboratory routine.

Keywords: Laboratories/standards; Basic research; Quality management; Certification/standards

INTRODUCTION

The management of research activities, from a strategic perspective, may add value to the organization because it results in generation of knowledge, transfer of results to society, research funding, and reduced operating costs⁽¹⁾.

Implementing a Quality Management System requires planning, document preparation, human resource training in the accreditation process, and internal audits, in order to evaluate quality based on established policies and goals. In addition, process organization and flexibility are required during the implementation to enable continuous evaluation and improvement of products and/or processes⁽²⁾.

For a system to be functional it has to be open and dynamic, and to have its complexity measured by the diversity of products and services offered, the number of people involved, the number of processes to be managed, the indicators and documents employed to measure and standardize processes⁽³⁾. The implementation of a quality management system, albeit time consuming and expensive, brings numerous advantages to the organization, both internally and externally, such as improved image of the company before its clients⁽²⁾. Furthermore, it offers advantages to managers, staff and clients, such as standardization and better control of processes, improved ability to detect and correct mistakes, and greater analytical reliability, respectively⁽⁴⁾.

According to the International Organization for Standardization (ISO) 9001:2008⁽⁵⁾, the Plan-Do-Check-Act (PDCA) tool may be useful in the implementation of a Quality Management System. This tool is employed in project management and improvement because it allows for identification of project objectives, goals and method (plan), training of the personnel involved and implementation of the plan (do), analysis of the results, comparing them against the proposed objectives and goals (check), improving methods, objectives and goals to optimize the project based on the defined objective (act)⁽³⁾.

In Brazil, Good Laboratory Practices (GLP) guidelines are described in the NIT DICLA 035 norm, which defines the GLP as a "quality control system that includes the organizational process and the conditions in which non-clinical health studies, as well as environmental safety studies are planned, developed, monitored, recorded, archived and reported"⁽⁶⁾. GLP requirements refer to adequate planning, good performance of quality control procedures, accurate documentation of all observations, adequate monitoring of activities and thorough archiving of all raw data, in order to eliminate errors⁽⁷⁾. GLP guidelines must be applied to safety assessment assays of all nonclinical tests of pharmaceutical products, pesticides, cosmetics, veterinary drugs, food additives and industrial chemicals.

It is important to note that the implementation of the NIT DICLA 035⁽⁶⁾ does not characterize the certification of the laboratory, but its accreditation to perform a specific activity. The accreditation process is voluntary and represents a formal acknowledgement that a laboratory or organization is capable to develop specific activities according to pre-established requirements. In Brazil, accreditation is granted by the Instituto Nacional de Metrologia, Qualidade e Tecnologia - INMETRO [National Institute of Metrology, Quality and Technology]. The certification process, on the other hand, denotes "a procedure that aims to provide an adequate degree of confidence in a certain product by means of compliance with requirements established by technical norms or guidelines"⁽⁶⁾.

GLP implementation involves two critical aspects that must be considered: firstly, it consists of guidelines that require interpretation during execution, and not of rules (which are found in other norms); secondly, these guidelines demand continuous improvement and advancement of technical and scientific knowledge, in order to ensure quality⁽⁸⁾. The present study evaluated the feasibility of implementing a Quality Management System in a basic research project, conducted in a laboratory of a public research and development organization.

OBJECTIVE

To evaluate the process of implementation of a Quality Management System in a basic research laboratory of a public institution and to verify its feasibility; to describe the impacts of the implementation process according to specific indicators; and to analyze the researcher's perception of the Quality Management System.

METHODS

Study type

This is a prospective qualitative study, executed within a public research and development institution, which aimed to evaluate the feasibility and impact of the implementation process. The Laboratory of Immunoparasitology (LIP) studies the immunopathogenesis of leishmaniasis, an insect vector-borne infectious disease. The LIP is located within the Centro de Pesquisa Gonçalo Muniz - CPqGM [Research Center Gonçalo Muniz)], which is one of the regional branches of the Fundação Oswaldo Cruz - FIOCRUZ [Oswalvo Cruz Foundation], an affiliate of the Ministry of Health.

Selection of applicable norms

Given that there are no norms designed exclusively for Research Laboratories, the NIT DICLA 035⁽⁶⁾ - GLP principles, was selected because it fulfilled preestablished criteria. Recommendations about GLP⁽⁹⁾ principles created by the Organization for Economic Co-operation and Development (OECD) were also used as complementary documents.

Phases of the implementation system

The PDCA tool was used⁽³⁾. The project was developed between January and December 2010 and divided into phases, as described below.

During the first phase (Planning), which took place in January 2010, laboratory policies, objectives and schedules were developed. Also during this phase a project was selected and designed according to the NIT DICLA 035⁽⁶⁾, which determines that its criteria be applied to pre-planned studies. The project "Investigation of the prevalence of leishmaniasis in asymptomatic blood donors in the city of Salvador" was selected because of its importance to public health and ease of execution, especially regarding assays. Indicators for evaluating the impact of the implementation of a Quality Management System were also defined (Table 1).

The second phase (Implementation/Execution), between February and September, involved training of the study team members, adapting the laboratory to comply with NIT DICLA 035⁽⁶⁾ requirements, and preparing, reviewing and approving of necessary documents. The selected project, namely "Investigation of the prevalence of leishmaniasis in asymptomatic blood donors in the city of Salvador", was executed during this time period.

During the third phase (Evaluation), extending from October to December, data were collected regarding compliance to the norms and team member's perception of the Quality System. Obstacles encountered by team members were also recorded. In order to evaluate team member perception of the Quality System, a questionnaire was administered to 20 people (10 researchers and 10 students). All team members were instructed about the nature of the investigation and signed an inform consent form. This project was approved by the local Research Ethics Committee, protocol number 237/2010.

The last phase proposed by the PDCA tool (Improvement) will be developed during future studies, based on the results of the present study.

RESULTS

The Planning phase was created by the project team members and approved by the laboratory heads. The NIT DICLA 043⁽¹⁰⁾ - GLP principles applied to project management across multiple sites (Multi-Site)" was added to the study design because sample collection occurred at a different institution. Indicators were analyzed after project conclusion (Table 1). The results show that there were no minutes recorded for meetings between the study director and the project team during the period in which NIT DICLA 035⁽⁶⁾ was implemented. Albeit non conformities were recorded, there was no movement towards correcting these. Furthermore, the institution remains without an equipment management policy in regards to equipment calibration and preventative maintenance. Document preparation was the only guideline requirement that was fulfilled. Accordingly, documents were prepared and utilized during project execution.

During the second phase (Implementation/Execution) project team members were trained and the laboratory adapted to comply with NIT DICLA 035 requirements. Information gathered during this diagnostic assessment enabled the identification of the main areas of improvement in the laboratory (Table 2).

Among the eight requirements evaluated only "Management System" and "Acquisition are not dictated by the NIT DICLA 035⁽⁶⁾. Nonetheless these requirements were maintained because of their important role in Quality Management. The diagnostic assessment performed before the implementation showed that compliance for all requirements was below 50%. The lowest compliance was observed for "Document Control", which showed the greatest improvement reaching 80% compliance after implementation. No changes were noted for "Raw Data" even after the implementation process. All necessary documents were prepared according to a standard, hierarchical template (Chart 1).

The research project was executed according to the study design and within the framework of the selected guidelines; by the end of the study, a final report was prepared following recommendations set forth by the NIT DICLA 035⁽⁶⁾.

During the Evaluation phase data were collected regarding compliance to the guidelines, team member's perception of the Quality System, and obstacles encountered by team members during the implementation. In order to evaluate team member perception of the Quality System, a questionnaire was administered to 20 people (10 researchers and 10 students). The best compliance rate was noted for "Document Control", at 100% compliance (Figure 1), with the lowest compliance noted for "Raw Data Control", which proved to be the main improvement point. Ours results demonstrate that, albeit the implementation process was not completed, positive changes have occurred. For example, operating procedures were prepared, equipments were calibrated, and equipment usage records implemented. This set of measures ensures traceability of equipment usage and prevents misusage. Reagent controls also improved significantly after implementation of a more rigorous monitoring system, as well as proper identification, labeling and storage. A non-conformity form was implemented to enable systematic annotation of both the problems identified and the corrective measures adopted in each case.

The survey used to evaluate the team's perception of the quality system showed that there is a clear understanding of the definition of Quality. Record keeping in the correct format, execution of experiments following SOPs and equipment calibration and maintenance were pointed as ways to ensure Quality. Among all of those who answered the questionnaire, 50% confirmed having used quality control tools for some time, starting before joining the CpqGM, while the other half stated starting to use such tools after joining the organization. Additionally, 75% agreed that a Quality System is important to ensure that the project progresses regularly, and that results are achieved. We also found that 91% of the people who were interviewed had never have their work or publications halted due to lack of a Quality System, which indicates that this is still an incipient theme in laboratories. When asked if they knew how a Quality System is implemented, 66% admitted to not having knowledge of the process. Lastly, 75% of the interviewees who answered the questionnaire demonstrated interest in learning more about Quality, either through training sessions or lectures. These data show that both researchers and students had sufficient knowledge about the theme and that they recognized the importance of this Quality System process (Tables 3 and 4).

The third and final phase of Evaluation, which consisted in recording the difficulties met by the team during the Planning and Execution phases of the GLP system implementation, showed that the main obstacles consisted of the paucity of information about the theme, and the fact that there are no guidelines specifically tailored for research laboratories, which created the need to adapt the NIT DICLA 035⁽⁶⁾ to the selected project.

DISCUSSION

The implementation of a Quality Management System, regardless of the setting, implies dedication, availability and effort on the part of everyone involved in the process.

In basic research this process remains infrequent. In parallel, research funding organizations in Brazil have yet to demand proof of competency from the research laboratories, or fulfillment of quality control requirements. However, in the United States and in European and Asian countries, financing is contingent upon such evidence⁽¹²⁾.

It is important to note that this work is the result of an innovative proposal within the scope of science and technology management, with the implementation of a quality system in a basic research laboratory in the health sciences. According to Nehme⁽¹³⁾, the mere choice of this theme is the result of a search for improved products, processes and management practices, and for excellence in organizations.

However this innovative movement occasionally meets resistance and difficulties. Early in the process, during the selection of the guidelines to be used, as well as during its implementation, we recognized the scarcity of literature on the subject in Brazil. Other groups alluded to such scarcity and also encountered resistance towards implementation of a Quality Management System within the laboratories⁽¹⁴⁾. There are reports which discuss the limitations imposed by the majority of guidelines, which are not tailored for research, and the need for increased flexibility when employing the existing norms⁽⁹⁾.

A very positive outcome of this work was the implementation of the traceability of data to all projects developed in the laboratory. Another important positive impact is that the laboratory is currently able to initiate a process since the staff is now capable of distinguishing between its essential and superfluous aspects.

Felicio⁽¹⁵⁾, in a similar investigation, reported encountering several obstacles: lack of a management culture, which transpired in people stalling in the concept of quality with a focus on inspection; lack of direction from the top management; and the nomenclature difficulty, since the people involved generally did not comprehend that the word "product", defined within guidelines, may also refer to services (which is the case for the Research and Development Center), among others.

In the present investigation the most important difficulty encountered was the lack of models or templates to be followed, for example, during the elaboration of the study design. Because research results and observations cannot be previously defined or predicted with precision, and can diverge significantly from initially expected outcomes, it is important to adopt flexible guidelines, and this also presented a problem. It is also important to emphasize the fact that this project was developed within a public institution, which follows an established routine and its own set of rules, among them are purchase bidding processes, approval through a chain of command, and routine internal audits performed by a trained team. All of these controls and inspections may contribute to the continuous improvement of ongoing processes.

Regarding the research processes adopted within the CPqGM, the NIT DICLA 035⁽⁶⁾ may be satisfactory and applicable if its guidelines and those of complementary set of norms are adapted to a research routine. However, it is still of utmost importance that a set of customized norms be created for research settings as to allow for the flexibility and originality essential for research and development activities. Researchers also need training on how to adapt the existing Quality Management norms to non-routine, original experiments.

Another great challenge is convincing the researchers, through training sessions, that a process such as this has many benefits. Establishing a Quality Management System in research and development, or in any other setting, is only possible if all of those involved believe in its advantages. Felicio⁽¹⁵⁾ described that only 70% of the researchers (both PhD and Masters) interviewed believed that implementing a Quality Management System would result in bureaucracy and unnecessary paperwork instead of adding value to research. This observation points towards the need for further clarification of the concepts and training directed to his public.

Importantly, the general quality management concepts are not novel among scientists⁽¹⁶⁾. What may be new is the possibility of integrating the quality intrinsic to research experiments to the controls dictated by a Quality Management System, aiming towards truly solid work. Finally, it is also important to emphasize that the GLP practices are guidelines, and not rules, with the objective of ensuring the quality and validity of data generated in assays. Above all, the main goal of such system is to facilitate the acceptance of the data both nationally and internationally. Nonetheless the responsible authorities must evaluate the intrinsic need for interpretation during implementation⁽⁸⁾. Besides, this is a dynamic system which requires continuous technical, scientific and operational improvements. Learning this concept is likely the main advantage of this work since there are many obstacles and possible interpretations that should be considered during the implementation of a GLP system. Therefore, the implementation of a GLP Quality System in a basic research laboratory is feasible, both economically and structurally, but is contingent upon training the existing human resources or incorporating new personnel.

CONCLUSION

We conclude that the implementation of a GLP Quality System in the Laboratory of Immunoparasitology (LIP, FIOCRUZ - Bahia) is feasible, both economically and structurally, but is contingent upon training the existing human resources or incorporating new personnel. Furthermore, we found that the implementation process had a positive impact on the project and on the research team.

Received on: Mar 17, 2012

Accepted on: Aug 9, 2012

Conflict of interest: none.

Study carried out at Laboratório de Imunoparasitologia do Centro de pesquisas Gonçalo Moniz, Fundação Oswaldo Cruz - FIOCRUZ, Salvador (BA), Brazil.

• 1. Schlatter PR, Bordin R, Matte US. Perspectiva estratégica em gestão na área da saúde. Revista HCPA. 2006;26(2):24-31.
• 2. Benoliel MJ. Step-by-step Implementation of a quality system in the laboratory. Trends Anal Chem. 1999;18(6):632-8.
• 3. Andrade FF. O método de melhorias PDCA [tese]. São Paulo: Universidade de São Paulo; 2003.
• 4. Almeida JS, Pires AC. Acreditação: Vantagens e dificuldades da implementação de um Sistema da Qualidade em um laboratório de ensaio e/ou calibração. Bol Soc Portuguesa Química. 2006;101:34-9.
• 5. Associação Brasileira de Normas Técnicas (ABNT). ABNT NBR ISO 9001: 2008 - Sistema de Gestão da Qualidade - Requisitos Gerais [Internet]. [citado 2010 Dez 16]. Disponível em: http://www.abnt.org.br
• ⁶
  Instituto Nacional de Metrologia, Qualidade e Tecnologia (INMETRO). INMETRO - 2009. Norma NIT-DICLA Nº 035. Princípios das Boas Práticas de Laboratório BPL. Julho 2009 [Internet]. [citado 2010 Jan 28]. Disponível em: http://www.inmetro.gov.br
• 7. World Health Organization (WHO). Good laboratory practice (GLP): quality practices for regulated non-clinical research and development. 2a ed. Genebra: WHO; 2009.
• 8. Brunetti MM. Critical aspects in the application of the principles of good laboratory practice (GLP). Ann Ist Super Sanità. 2002:38(1):41-5.
• ⁹
  Organisation for Economic Co-operation and Development (OECD). OECD 1998. The OECD Principles of good laboratory practice [Internet]. [citado 2010 Mar 13]. Disponível em: http://www.oecd.org
• ¹⁰
  Instituto Nacional de Metrologia, Qualidade e Tecnologia (INMETRO). Norma NIT DICLA Nº 043. Aplicação dos princípios de BPL à organização e ao gerenciamento de estudos em múltiplas localidades (Multi-site). Fevereiro 2010. [Internet]. [citado 2011 Mai 18]. Disponível em: http://www.inmetro.gov.br
• 11. Associação Brasileira de Normas Técnicas (ABNT). ABNT NBR ISO 17025: 2005. Requisitos gerais para a competência de laboratórios de ensaio e calibração. Rio de Janeiro; ABNT; 2005.
• 12. Robins MM, Scarll SJ, Key PE. Quality assurance in research laboratories. Accred Qual Assur. 2006;11(5):214-23.
• 13. Nehme NS. Implantação do sistema de gestão da qualidade em um laboratório de pesquisa do Instituto Oswaldo Cruz (IOC): Desafios e soluções da rastreabilidade do programa PALC (Programa de Acreditação de Laboratórios Clínicos) da SBPC/ML (Sociedade Brasileira de Patologia Clínica - Medicina Laboratorial) [tese]. Rio de Janeiro: Fundação Oswaldo Cruz. Escola Nacional de Saúde Pública Sergio Arouca - ENSP; 2008.
• 14. Bordini ME. Implantação de um sistema de gestão da qualidade em laboratórios de pesquisa em saúde: planejamento, viabilidade e impacto do caso em uma organização pública de pesquisa e desenvolvimento [tese]. São processo de implantação sobre indicadores selecionados [tese]. São Paulo: Paulo: Universidade de Taubaté; 2008. Universidade Federal de São Paulo; 2009.
• 15. Felicio D. Implantação de um Sistema de Gestão da Qualidade: Estudo de caso em uma organização pública de pesquisa e desenvolvimento [tese]. São Paulo: Universidade de Taubaté; 2008.
• 16. Mathur-De-Vré. The scope and limitations of QA system in research. Accred Qual Assur. 2000;5(1):3-10.

Publication Dates

• Publication in this collection
  22 Jan 2013
• Date of issue
  Dec 2012

History

• Received
  17 Mar 2012
• Accepted
  09 Aug 2012