Quality perception in research laboratories from Fiocruz after QMS implementation

Ivanete Milagres Presot Oswaldo Cruz Foundation (Fiocruz), René Rachou Research Center Rodrigo Pedro Pinto Soares Oswaldo Cruz Foundation (Fiocruz), René Rachou Research Center Ana Paula Madureira Federal University of São João Del Rey Kelly Alves Bicalho Oswaldo Cruz Foundation (Fiocruz), René Rachou Research Center Federal University of Ouro Preto Celina Maria Modena Oswaldo Cruz Foundation (Fiocruz), René Rachou Research Center


Introduction
Quality Management Systems (QMS) was originated in industry and have been expanded to other fields, being an important key of globalization.It is associated with a style of management in which a systemic view and continuing improvement combine with the aim of increasing performance (Vieira et al., 2011).Quality Management Systems worldwide share common requirements, but comprise unique characteristics depending on the context (Presot and Silva, 2006).Among its definitions, quality can be understood as an essential and differential attribute of a product or as a service satisfaction and commitment (Crosby, 1990).According to Juran's trilogy, quality consists of planning, control, and improvement (Juran, 1990), while Deming (1990), defined quality based on the PDCA cycle (Plan, Do, Check, Act).
Total Quality Management, optimized in Japan after World War II, was established in Brazil in the 1980s (Falconi, 1990), followed by the creation of many awards including the National Quality Award (NQA) and the Federal Government Quality Award (FGQA).In 1987, the International Organization for Standardization (ISO) published the 9000 series for QMS to standardize the exchange of products and services.The most recent review of ISO 9001 was in 2008(ABNT, 2008).Although ISO 9000 is widely adopted in global business its implementation is voluntary.
The trend toward QMS inclusion in business was followed worldwide by academic centers concerned with the verification and reproducibility of scientific research (Silva, 2007).In the 1970s, irregularities in toxicity laboratories were detected by the United States Food and Drug Administration (FDA).This resulted in the development of the Good Laboratory Practices (GLP) manual for non-clinical safety studies.In 1981, the Organization for Economic Co-operation and Development (OECD) published GLP principles, adopted in 1986 by the European Union (OECD, 1998).Those practices are currently widespread on a global scale.Good Laboratory Practices was first imposed on industry by regulatory authorities, followed by Good Manufacturing Practices (GMP) and Good Clinical Practices (GCP) (WHO, 2006).In Brazil, GLP is mandatory in non-clinical laboratories handling pharmaceuticals, agrochemicals, cosmetics, veterinary products, food additives, food, and industrial chemicals (Inmetro, 2009).Other available guidelines include ISO 17025 for testing and calibration.The most recent review of this standard was in 2005 (ABNT, 2005).
Clinical diagnostic laboratories follow ISO 15189, published in 2003and revised in 2008(ABNT, 2008a).According to Plebani (2003), the pursuit of accreditation is crucial to the improvement of laboratory services and to move toward an international consensus.In this sense, laboratory medicine can be considered a pioneer in quality promotion (Vieira et al., 2011).Ovretveit (2013) also reinforced the discussion of quality in the public health and in health services pointing an increase in the use of indicators and the importance of quality in the research performed by nurses, physicians and project leaders.Knowledge of research methods, especially the collection and interpretation of data are essential tools for successful results.
Quality is essential in research centers and universities, and, since these institutions are not covered by GLP, it is necessary to implement QMS.Researchers and sponsors have questioned the use of traditional peer review system as the sole means for research evaluation, emphasizing the need for quality principles in these institutions (Robins et al., 2006).Since basic research is not covered by GLP, in 2006 the TDR/WHO published the Quality Practices in Basic Biomedical Research Handbook (QPBR) (WHO, 2006).
In Brazil, the publication of ABNT 16501 (Guidelines for research management systems, development and innovation) by the Brazilian Association of Technical Standards (ABNT) in December 2011 represented an important step for standardization of research, development, and innovation (RD & I).Although it was based on ISO 9001, its focus is not directly relevant to universities and research center laboratories (ABNT, 2011).Therefore, the São Paulo Research Foundation (Fapesp) has published the Guide of Good Scientific Practice, dealing with ethical guidelines in conducting scientific research (Fapesp, 2011).

The Oswaldo Cruz Foundation and the René Rachou Research Center
The Oswaldo Cruz Foundation (Fiocruz) is part of the Brazilian Health Ministry, and is one of the most important research institutions in Latin America.Its mission includes promoting health and social development, in addition to generating and disseminating scientific and technological knowledge.For more than a century, Fiocruz has dealt with the major public health problems of Brazil, becoming a center of excellence in biomedical research.Its mission expanded to outpatient hospital services, reference laboratories, and the production of vaccines, drugs, reagents, and diagnostic kits as well as education and training, information and communication, quality control of products and services, and implementation of social programs.Fiocruz was established in Rio de Janeiro in the beginning of the twentieth century and currently has several regional sites, including the René Rachou Research Center (CPqRR/ Fiocruz) in Belo Horizonte, Brazil.
The CPqRR/Fiocruz (www.cpqrr.fiocruz.br) is a reference institution for public health research.Its mission is to provide a better life quality based on health care policies, research, technological development, innovation, teaching, and reference services.It consists of 14 laboratories with modern infrastructure allowing the study of cellular and molecular biology, epidemiology, immunology, and the search for new drugs and vaccines for tropical diseases, including Chagas disease, leishmaniases, malaria, schistosomiasis, and intestinal parasites.Besides infectious disease, research is devoted to epidemiological aspects on aging, risk behavior, and occupational diseases.In 2003, a graduate course on health sciences was established at the institution to focus on training students (Master and Ph.D.) for scientific research, teaching, and professional activities in the health field.Additionally, CPqRR provides scientific and technical support for the Brazilian Public Health System (SUS).
In 2003, Fiocruz issued Internal Regulation 430/2002 requiring the implementation of QMS in all reference laboratories.The CPqRR required the implementation of NBR ISO 17025 in its reference laboratories.Some projects of technological development adopted GLP.This was followed by the research laboratories, which lacked specific QMS regulation.In 2006, WHO published the QPBR handbook (WHO, 2006).This guide was developed by a group of specialists, including researchers, and focused on relevant and practical aspects of laboratory routine.Fiocruz has been investing in QMS implementation as exemplified by use of the GLP in technological platforms of the Aggeu Magalhães Research Center (CPqAM/Fiocruz), a regional site of Fiocruz in Recife, Brazil.This reflects a recent interest in the implementation of QMS, not only in Fiocruz, but also for research and development (R & D) institutions.This aims to comply with national and international regulations, an important requirement for certification/accreditation and sometimes necessary for sponsoring agencies (Souza et al., 2012).Another example is a survey that evaluated GLP implementation in a basic research laboratory in the Gonçalo Muniz Institute, demonstrating the viability and positive impacts in the laboratory routine (Fraga et al., 2012).
In the past ten years, QMS has been implemented in the reference and research laboratories of the CPqRR/Fiocruz.During its implementation a group of Quality Coordinators was trained to disseminate QMS culture into laboratories.
After this period, the question of how QMS affected the collaborators (Cs) perception arose as a major concern of the institution's Office of Quality and Biosafety (OQB).

Quality perception studies
Perception studies are relevant to understanding unknown aspects of the organization and development of improvement measures (Vaitsman et al., 2003).Perception analyses are widely used in health services.Castellanos (2002) conducted a descriptive study using survey methods and a structured questionnaire in order to assess the customer's and the staff's perceptions in the Hospital of the University of São Paulo.The results demonstrate that the quality perception was different among the two groups.The theoretical framework used by this researcher had a strong contribution of the marketing field, including quality attributes, field methodology and quantitative analysis.More recently, Fadel and Filho ( 2009) compared clients' and dentists' perceptions of a clinic in the city of Florianopolis, Brazil.This study showed that technical quality was important to the professionals, while interpersonal qualities were more relevant for the patients.The perception evaluation of internal clients offered a reliable tool enabling the detection of improvement opportunities in internal processes.Santos and partners (2008) reported the effectiveness of social responsibility programs from a banking institution.In this survey, the perception of public employees resulted in changes in the current programs indicating new areas of action.There are currently no available reports of perception evaluations about QMS in research laboratories.

Objective
This study aimed to analyze the professionals' perception of quality in research laboratories of the René Rachou Research Center (CPqRR/Fiocruz) after ten years of QMS implementation.

Methodology
The survey was performed in 12 research laboratories of the CPqRR/Fiocruz regional center of Fiocruz located in Belo Horizonte, Minas Gerais, Brazil.All laboratories involved participated in QMS implementation.The project was approved by the Ethics Committee of CPqRR/ Fiocruz (protocol #22/2010).This was an applied, quantitative and descriptive field research.Data collection investigated the quality perception of the laboratory collaborators.The questionnaires were applied to the participants together with an informed consent form.The structured questionnaire was used in order to expose all participants to the same stimuli, open and closed questions, and the same procedures for data collection.The way to collect the data was self-administered while preserving the anonymity of participants.The questionnaires were developed based on the quality parameters defined by the QPBR handbook (figure 1).The concepts and interpretation of quality parameters presented in figure 1 can be also found in the QPBR handbook, officially translated by CPqRR (WHO, 2010).
The total population of these laboratories is ~350 people.A group of 113 collaborators was interviewed (n=113) including researchers, technicians, lab managers, project managers and students, representing 30% of laboratory collaborators.Names of individual interviewees were not recorded to ensure information confidentiality.The first part of the questionnaire required the evaluation of the quality system importance in a research laboratory with respect to several parameters (figure 1), using the Likert scale with five possible responses: 1) strongly disagree, 2) disagree, 3) neither agree nor disagree, 4) agree, and 5) strongly agree.
The questionnaire also addressed areas that could be improved in the institution.In this question, the collaborators had the opportunity of choosing up to three options among 13 parameters assessed in the first part of the questionnaire.Finally, in the last part the collaborators had the opportunity to include comments about QMS.
The majority of answers were in the "agree" and "strongly agree" columns for all parameters (graphic 1).The higher frequency (88-99%) of responses in the "strongly agree" column was obtained for three parameters: biosafety, training and ethics.The frequency of responses in the "strongly agree" column was between 60 and 70% for data recording, study plan, SOPs, file storage and supervision and quality assurance.The lower frequency (50-56%) of responses in the "strongly agree" column was obtained for result report, result verification, personnel (Job descriptions) and publishing practices.The parameters that needed most investment were: training, biosafety and quality policy/staff responsibility.

Perception of quality parameters importance by collaborators (Cs) from the laboratories of René Rachou Research Center/Fiocruz
Source: Elaborated by the author.

Ivanete Milagres Presot • Et. al
The sentences below provide comments made freely by the surveyed collaborators: The system is extremely important to increase the quality of the research developed in research laboratories.However, it should not interfere with the research, an extremely dynamic and fast process.
An Ethics discussion should be included in the Institution's meetings.Ethics in public service ... how to behave in a public institution.
The awareness of employees about the importance of equipment handling and the workplace.
It is important to encourage the staff to engage in the quality strategies, the quality week is a big event, but it only happens once a year.
Each laboratory should have at least one permanent employee responsible for quality practices.
The current guidelines do not specify the storage period of research records, making difficult to establish the time for file storage.
The next step was to analyze the data according to professional category in the Cs group.The parameters biosafety and training showed the highest agreement among professional categories at 93.3 and 88.2%, respectively.A high level of agreement was also observed for the parameter ethics, except by undergraduate students (66.7%) (figure 2).
Personnel (job description) and recording data exhibited a similar profile, with managers having the highest percent agreement with the statements (figure 4).Finally, SOP showed similar levels of agreement among the professional categories except for master's students with a 100% of agreement (figure 4).
Competition for funding is increasing, and it is essential that scientific research, especially in the health field, be conducted in order to reduce waste of resources and avoid unnecessary duplication of effort.Quality practices are crucial to define, plan, perform, record, monitor, and report research (Robins, 2006;WHO, 2006).According to Grouchau and partners (2010) factors for the successful implementation of QMS in research and development environments may include the acceptance and commitment of all professionals involved (the so-called "quality culture"), the development of a flexible documentation system, and, more importantly, the QMS must add value to the institution.The benefits of QMS implementation have been reported in the research field, especially in institutions with accreditation requirements.It is expected that accreditation may become a future requirement for publication in major scientific journals (Abad, 2005).
Frequency of "strongly agree" responses to the remaining parameters varied among the professional categories.In general, master's students and managers exhibited the highest agreement with the statements, followed by Ph.D. students/technicians and undergraduate students/ researchers.This pattern was observed for quality policy, supervision and quality assurance, results verification, result reporting, publishing practices, study plans, and file storage (figures 3 and 4).In this study, it was evaluated the QMS collaborators perception in laboratories of a public research institution.High agreement was observed in the responses "agree" and "strongly agree", which may reflect the dissemination of quality culture and evidence that the investment in periodic QMS training was effective in the institution.The parameters training, ethics, and biosafety had high agreement scores in both groups (>88%) (graphic 1).Those factors are closely related to professional development and improvement of working conditions, probably understood as benefits for workers.According to Stashevsky and Elizur (2000), the perception by a given collaborator of his/her role in the decision-making process is important for better individual performance.Employees' perception that their work is meaningful has a crucial impact on their performance.

Ivanete Milagres Presot • Et. al
The parameters data recording, study planning, file storage, supervision and quality assurance, SOPs and quality policy/staff responsibility exhibited frequencies below 70% in the "strongly agree" column (graphic 1).Among the parameters with intermediate level (60-70%), the recording data was considered the most important.Although the recording books are well established in the Institution, they still need more investment during the QMS training.This was also reinforced by the Fapesp Scientific Practices, which states data research should be recorded in a concise and complete way.Additionally, the QPBR handbook emphasizes that study plans should be easily distinguished from the data in the register books (WHO, 2006).Concerning file storage, one collaborator identified the need for specifying how long those documents should be kept.Besides the discussion about storage time, it is important to reinforce that, according to QMS regulations, those documents should be located in a specific place, easily identifiable where the documents could be promptly retrieved.
For SOPs, the major difficulty in implementing QMS in scientific environments is related to the standardization and routine control.Standardization may hinder achievement of research objectives.Standardization, SOP and other components are perceived as limiting creativity, but it is important to remember that quality has two sides, technical and humanistic.The implementation of a system depends on the involvement of employees and represents more than just establishing procedures.There are no guidelines about how the QMS should be done, so its strategic implementation should address how to achieve satisfactory results (de Souza, 2012).Supervision and quality assurance besides results verification that is important to ensure reproducibility and publication is also related to the study inspection.In the case of research laboratories, those inspections are not simple tasks, due to the variety of projects and procedures carried out by researchers, undergraduate, and graduate students.According to WHO (2006), to avoid conflict of interest, audits should be conducted by outside professionals, or by someone in the institution who is not directly involved in the study.
QMS implementation and maintenance requires a continuous effort from the Institute Direction providing the infrastructure and human resources necessary for it (Juran, 1990).More importantly, leadership comes from top management and includes individuals and teams aiming commitment, integrity, honesty and truth.This need was clearly demonstrated during the interviews, in which the collaborators identified the need of specific permanent employees for QMS activities.
The QMS parameters rated with the lower scores by the collaborators were: result reporting, result verification, personnel (job description) and publishing practices (values below 56.0%) (graphic 1).Those data may indicate the need for specific training by the office of quality and biosafety in those areas.This is of importance, since result verification, result reporting, and publishing practices are strategic for the research project framework.In addition, the validity of scientific research depends on well-planned schedule, execution, and documentation (WHO, 2006).
The second area of the questionnaire aimed to identify the parameters perceived by Cs that need further investment.Training, biosafety, and quality policy/staff responsibility were the most important parameters.Training and biosafety responses overlapped with those with Rev. Adm. Pública -Rio de Janeiro 48(1):237-52, jan./fev.2014 highest agreement scores in the "strongly agree" column (graphic 1).This emphasizes the need for specific training for personal and professional development to avoid accidents and hazards in research laboratories.In CPqRR/Fiocruz the collaborators are exposed to microbial pathogens, laboratory animals, and chemical substances that are potentially hazardous.With regard to the Quality Policy, the concerns of employees regarding the motivation are implied in the following statements: "...Each laboratory should have at least one permanent employee responsible for quality practices"."...It is important to encourage staff in the quality strategies engagement.The quality week is a big event, but it only happens once a year." When analyzing the quality parameters per collaborator professional category, again biosafety and training did not exhibit differences among professions (figure 2).However, substantial differences were observed for the remaining parameters (figures 3 and 4).The highest agreement rates were observed for master students and managers for quality policy, supervision and quality assurance, result verification, result report, publishing practices, study planning, and file storage.With the exception of training and biosafety, the lowest levels of agreement were observed for undergraduate students and researchers.These students are part time and less involved with the institution quality policies, thus, specific QMS training in this group should be reinforced, as the students usually enter graduate school and remain at the institution for the subsequent years.The lower ratings observed for researchers may reflect their motivation towards QMS practices.While they recognize the importance of its implementation in the laboratory, they usually elect a QC to be in charge.This may reflect the motivation of researchers, being more involved in scientific matters in the institution.Ortner (2000) reported that leadership behavior plays a decisive role in QMS implementation.Consistent with our results, initiatives to implement QMS in university laboratories and research centers face difficulties depending on the category (Abad, 2005;Grouchau et al., 2010).
Furthermore, the category of master's students showed higher levels of "agree" responses than Ph.D. students, who in turn presented a higher rate than the researchers.As reported by Souza and partners (2012), Fraga and partners (2012) and Camman and Kleibohmer (1998), scientists in some research centers, specifically those with an academic profile, show concern for and resistance to, not only the activities related to the implementation of the system, but also the consequences generated by the modifications and proposals for adjustments.Some argue that the rigidity of a formal QMS and excessive rules can stifle the scientific progress, reduce creativity, and increase bureaucracy.This was clearly demonstrated in the following sentence from a collaborator: "...The system is extremely important to increase the quality of the research developed in research laboratories.However, it should not interfere with the research, an extremely dynamic and fast process."However, in CPqRR there was a significant improvement in the quality culture.This result may be related to attributes of Ph.D. students and researchers who are overwhelmed with experiments, publishing, mentoring of students, preparation, and submission of grant projects, participation in conferences, and management activities.The excess of activity hinders motivation and participation in training activities of QMS.Therefore, alternative ways of attracting those professional categories should be implemented by OQB.According to Fraga Ivanete Milagres Presot • Et. al and partners (2012), an important challenge is to convince the researchers of the benefits following the implementation of QMS and suggest training activities as a tool to achieve this objective.

Conclusions
This study demonstrated that QMS was successfully implemented in the laboratories of the CPqRR/Fiocruz indicating that a QMS culture was established.Perception of a quality parameter was shown to vary according to professional category.These data reinforce the need for a continuing QMS maintenance program to provide specific training to the groups in which differences were detected.Understanding the perception of collaborators allows the development of improvement actions, directing strategies for educational guidelines and for disseminating quality.

F
i g u r e 1 Quality parameters evaluated according to the Handbook of quality practices in basic biomedical research (WHO, 2006) Source: Elaborated by the author.

F i g u r e 2
Differential perception of collaborators (Cs) from the laboratories of Centro de Pesquisas René Rachou/Fiocruz regarding biosafety, training, and ethics Source: Elaborated by the author.Note: Underg.: Undergraduate student; Master: Master student; Ph.D.: Ph.D. student; Techn.: Technician; Res.: Researcher.

F i g u r e 3
Differential perception of collaborators (Cs) from the laboratories of René Rachou Research Center/Fiocruz regarding quality policy, supervision and quality assurance, result verification, result report, publishing practices, and study planning Source: Elaborated by the author.Note: For abbreviations, see the subtitles of figure 2.

F i g u r e 4
Differential perception of collaborators (Cs) from the laboratories of René Rachou Research Center/Fiocruz regarding file storage, personnel (job description), data recording and standard operation procedure (SOPs)Source: Elaborated by the author.Note: For abbreviations, see the subtitles of figure 2.