Safety evaluation in vaccine care: elaborating and validating a protocol

Abstract

Objective:

to elaborate and validate a protocol for evaluating the safety of nursing care with vaccines during primary care.

Methods:

methodological research developed in two stages: protocol elaboration and content validation; and using the instrument, through the Delphi technique. The evaluation of the instrument was calculated by the Content Validity Coefficient, considering items above 70% of concordance among judges.

Results:

the instrument obtained a concordance index for the eight questions analyzed in the first stage. Its application in the practice of primary care services was recommended by 75% of the judges in the second Delphi stage.

Conclusion:

the protocol for safety in nursing care with vaccines proved a high credibility and its adoption in health institutions can contribute to the quality of vaccine care and the conduct of professionals.

Keywords
Vaccines; Nursing care; Patient safety; Validation studies; Protocols

Resumo

Objetivo:

Construir e validar um protocolo para avaliação do cuidado seguro de enfermagem com vacinas na atenção primária.

Métodos:

Pesquisa metodológica desenvolvida em duas etapas: construção do protocolo e validação de conteúdo e aparência do instrumento, através da técnica Delphi. A avaliação do instrumento foi pelo cálculo do Coeficiente de Validade de Conteúdo, tendo sido considerados válidos os itens com mais de 70% de concordância entre os juízes.

Resultados:

O instrumento obteve um índice de concordância para os oito quesitos analisados já na primeira rodada e sua aplicação na prática dos serviços de atenção primária foi recomendada por (75%) dos juízes na segunda rodada Delphi.

Conclusão:

O protocolo para a segurança do cuidado de enfermagem com vacinas demonstrou alta credibilidade e sua adoção nas instituições de saúde pode contribuir para a qualidade da assistência com vacinas e das condutas dos profissionais.

Descritores
Vacinas; Cuidados de enfermagem; Segurança do paciente; Estudos de validação; Protocolos

Resumen

Objetivo:

Construir y validar un protocolo para la evaluación del cuidado seguro de enfermería con vacunas en la atención primaria.

Métodos:

Investigación metodológica desarrollada en dos etapas: construcción del protocolo y validación de contenido y apariencia del instrumento, a través de la técnica Delphi. La evaluación del instrumento fue por el cálculo del Coeficiente de Validez de Contenido, habiendo sido considerados válidos los ítems con más del 70% de concordancia entre los jueces.

Resultados:

El instrumento obtuvo un índice de concordancia para los ocho temas analizados ya en la primera ronda y su aplicación en la práctica de los servicios de atención primaria fue recomendada por el 75% de los jueces en la segunda ronda Delphi.

Conclusión:

El protocolo para la seguridad del cuidado de enfermería con vacunas demostró alta credibilidad y su adopción en las instituciones de salud puede contribuir a la calidad de la asistencia con vacunas y conductas de los profesionales.

Descriptores
Vacunas; Atención de enfermería; Seguridad del paciente; Estudios de validación; Protocolos

Introduction

Nursing is recognized for health care activities and, for the safe execution of its actions, knowledge that supports practice must be produced. In addition to scientific evidence, effective strategies contribute to prevent and mitigate risks in health facilities, many with technology subsidies.^(1,2)

Seen that, elaborating protocols becomes essential, since they represent a technological tool that provides systematic orientations to direct professionals and contribute to care. These are important characteristics to be considered for the adequate performance of standardized functions, with structured procedures and time optimization.^(3–5)

For this, protocols need to be based on scientific evidence and associate aspects of the literature and the context of care so that they enable organizing the actions and providing innovations for conducts. It is of utmost importance that the incorporation of these technologies provides quality to work performed and, within this context, the methodological rigor must be guaranteed.^(1,6,7)

Therefore, for the resource to have credibility and legitimacy, the validation process becomes essential, since it represents a factor of greater reliability for the selection and/or adoption of a protocol. Given that, content and appearance validation are the two most commonly used types for these instruments in Health.^(4,8)

In content validation, the items of the instrument are judged according to their relevance, representativeness and comprehensiveness. In appearance, the material is evaluated for clarity and presentation.⁽⁸⁾

Thus, the concept of validity is adopted when a tool can achieve what is proposed. Thus, Nursing must innovate and submit them to tests and judicious analysis through research.^(7,9)

Given that these resources constitute important elements in health services, their incorporation may favor improvements in actions, such as those of immunization. The emphasis on the need of this technology directed towards nursing care with immunobiologicals is stated, since the availability of the types of vaccines offered in the public network and the increasing number of doses administered can cause an increase in Adverse Events (AEs).^(10.11)

After immunization, reports of these events have been considered relevant worldwide, which requires monitoring safe vaccine care.^(10,11)

Associated with this, such deficiencies contribute to unnecessary losses, usually caused by failures in storage, transportation and/or manipulation, due to the lack of conservation of vaccines in Brazilian vaccine rooms. These situations consequently impair and inactivate the referred vaccines.^(10,12)

Therefore, this study aims to construct and validate a protocol for evaluating the safety of nursing care with vaccines during primary care.

Methods

Methodological research, with a quantitative approach, developed from September 2017 to February 2018, in two stages: 1. Elaborating the protocol; 2. Validating the content and appearance of the instrument by judges, using the Delphi technique.

Elaborating the protocol

For this stage, results from the literature review were used, through a scoping review, based on the Brazilian legal framework through consultations with Brazilian ministerial manuals: the Manual for Vaccination Norms and Procedures (Manual de Normas e Procedimentos para Vacinação) and the Manual for the Cold Chains of the National Immunization Program (Manual de Rede de Frio do Programa Nacional de Imunização) and, as well as national and international scientific evidence (^{Appendix 1}).

Validating the content and appearance of the instrument

This process was carried out through the analysis of judges selected for research, chosen intentionally, after reading their curricula in the Lattes Platform of the National Council for Scientific and Technological Development (CNPq). For this, the simple search form was used in the “search for” field in the “Subject” category, using the terms “patient safety” and/or “vaccines”.

The search found 94 experts that hold master's degrees and 68 experts that hold doctor's degrees. For the selection of possible judges, Fehring⁽¹³⁾ model was adapted and used, given it has a maximum score of 14 points. However, for this selection, a minimum score of 5 points was assigned and the first 30 who reached this average were chosen.

From this perspective, the eligibility criteria for including the judges were: hold a degree in Nursing; hold a postgraduate degree (Strictu Sensu) in Health; publish on patient safety and/or vaccines. Judges who did not answered the electronic questionnaire within the defined period did not confirm their participation, and/or those who did not return the Free and Informed Consent Term (TCLE) signed in the proposed period were excluded.

The initial contact with possible judges was made in the form of an invitation letter sent electronically, with explanations about the study. For those who agreed to collaborate, the TCLE was sent. After that, the protocol was sent via Google Forms.

The material addressed to judges had two parts: the first referred to the characterization of the specialists; the second contained instructions for analysis of the protocol, with three categories for evaluating the items: Suitable, Partially Suitable or Inadequate. This process was conducted by the Delphi technique. The specialists answered, in stages, an evaluation questionnaire.⁽¹⁴⁾

Of the thirty possible judges initially selected, twelve accepted to participate in the evaluation of the protocol, corresponding to the first stage (Delphi 1). They could suggest changes in the material for its improvement. Some of these changes were considered pertinent and, after adjustments, a feedback about the answers was sent together with the protocol, corresponding to the second stage (Delphi 2), in which eight judges participated.

For the protocol to be considered valid, the judges needed to evaluate the components of the protocol in both Delphi stages, respecting some criteria: behavior, objectivity/desirability, simplicity, clarity, relevance/pertinence, precision, typicity and amplitude.⁽¹⁵⁾

Analysis of results

Data were stored in an Excel spreadsheet and analyzed in a descriptive way. The evaluation of the protocol was performed by calculating the Content Validity Coefficient (CVC) ≥ 0.78 for the separate items and the protocol in general. The Appropriate/Partially Appropriate items with more than 70% agreement within the judges were considered valid.⁽¹⁵⁾

Ethical aspects

The study complies with the norms that involve ethical aspects in research with human beings, approved by the Research Ethics Committee under Opinion no. 1.768.233 and Certificate of Presentation and Ethical Appraisal (CAAE) n° 59962316.8.0000.5537.

Results

In the process of elaborating the protocol, a structured observation script was prepared, in a check-list format, containing seven items (vaccination room; control and recording of refrigerator temperature; attention to the refrigerated chamber; Nursing conducts in the vaccination room; organization of vaccines in the refrigerator; conducts with thermal boxes to store vaccines, and measures taken with vaccines under suspicion). Moreover, their respective subitems were observed, which represent the requirements of the protocol (^{Appendix 2}). Therefore, for evaluation, the check-list should be consulted to assist in the integral analysis of all the topics listed in the protocol.

After elaborating the protocol, validating it took place. Thus, the first Delphi stage was characterized by female people (100%; n=12), with a minimum age of 31 years old and a maximum of 57 years old (average=42.8), predominance of doctors (66.7%; (n=8), time to obtain the highest degree from three to six years (41.7%, n=5), with experience in teaching (91.7%, n=11) and services (66.7%; n=8) of Primary Health Care.

Delphi 2 was made up of eight judges. All were women aged between 31 and 54 (average=44.2), held a doctor's degree (62.5%, n=5), the time to obtain the highest degree ranged from three to six years (62.5%, (n=5), had experience in teaching (100%, n=8), and experience in the practice of PHC services (62.5%, n=5).

After Delphi 1, the suggestions given by judges about the seven items - and their respective subitems -, which make the check-list (prerequisites of the protocol) were evaluated. All recommendations were analyzed, and feedback of the responses was forwarded to the judges. The main items evaluated are shown in chart 1.

Chart 1
Summary of changes suggested by the judges in Delphi Stage I

Protocol Items	Aspects included from the suggestions given by the judges in Delphi 1
Vaccination Room	Is there ambient air conditioning between +18°C and +20°C? Is there any way to control the temperature inside the room? Is there a routine for verifying the validity of immunizations and supplies (syringes and needles)? Add a footnote clarifying what it is part of Group A1. Is there only a refrigerator in the vaccination room or is there also a cooled chamber?
Conditions of the Refrigerator	Is the sealing rubber of equipment tested?
Cooling temperature control and recording	There were no items added.
Conducts of Nursing in the vaccination room	Is the person responsible and/or person vaccinated receiving advise on the immunizations administered, possible occurrence of adverse events, or being investigated about the possibility of pregnancy, immunosuppressive diseases, allergies, and carrying out the next vaccine schedules? Is the vaccination registration requested to those responsible and companions during the care? Is the vaccinator using the technique of leaving a needle and syringe in the vial/ampoule for them to aspirate? Is the “right-nine” checklist done before administering the vaccine?
Organization of vaccines in the refrigerator	Are the opened vials identified with the opening date? Are vaccines packaged on the second and third shelves (for domestic refrigerators)? Is the first shelf not used to pack vaccines (for domestic refrigerators)? Are the diluents next to the lyophiles?
Conducts with thermal boxes	There were no items added

After the changes suggested by judges in Delphi 1, the final protocol presented the following elements: vaccination room - being exclusive to administer vaccines; having refrigerated internal temperature with air conditioning, and ambient air conditioning between + 18°C and + 20°C; existing a form to control the internal temperature of the room, some bureaucratic material for the expedient (booklets and forms used to record activities), and rooms with standards, technical and operational manuals available for consulting and clarifying doubts of the professionals; performing routines to verify the validity of immunobiologicals and supplies (syringes and needles); storing syringes and needles in a closed cabinet; disposing infectious residues (Group A1) and piercing/cutting material (empty immunological vials, syringes and used needles) from the vaccination room discarded in waste containers; having exclusive electrical plugs for each piece of equipment and circuit breakers for vaccination rooms with a warning not to be turned off; and supervising vaccination rooms daily by nurses.

Regarding the control and temperature record of the refrigerator, the elements are: having a professional responsible for reading the maximum and minimum thermometer, and a thermometer for maximum and minimum temperatures in refrigerators; using digital thermometers or an extension cable; performing the daily reading and recording temperatures at the beginning (before the first opening of the refrigerator door) and end of the working day (after the last door closing); filling in the daily temperature control map fixed in a visible location; adjusting the thermostat inside the refrigerator and; communicating to the superior instance in case of temperature change.

The third item, refrigerator care, had nine items, namely: having refrigerators/exclusive refrigerated chambers to conserve vaccines, refrigerator/refrigerated chamber in proper working conditions, and cooler/refrigerated chambers away from heat sources and sunlight; keeping at least 20 cm from the wall; performing tests for the rubber of equipment; defrosting and cleaning performed every 15 days or when the ice sheet reaches 0.5 cm in the refrigerator; having a reusable coil in freezer; putting bottles filled with water mixed with dye in the bottom drawer of the refrigerator; conducting a periodic preventive/corrective maintenance of refrigerators/refrigerated chambers.

As for nursing conducts in vaccination rooms, the items were: controlling storage temperature for vaccines (between +2°C and +8°C, being +5°C the ideal); conditioning vaccines that arrive at the unit in thermal boxes with a thermometer; doing the “right nine” checklist before administering the vaccine; providing guidelines to those responsible and/or the vaccinated on the immunobiological administered; for vaccinators, using the technique of leaving a needle and syringe in the vial/ampoule for them to aspirate; performing hand hygiene before and after the procedures; requesting the registration of vaccination to those responsible and companions during care.

The items for vaccines storage in the refrigerator were: organizing vaccines in trays; not using the first shelf to pack vaccines (for domestic refrigerators); putting vaccines in the second and third shelves (for domestic refrigerators), and diluents together with lyophiles; identifying open bottles with their respective opening date; keeping vaccines away from the walls of the refrigerator; organizing vaccines according to their type, lot and validity; putting products with a shorter shelf life at the front; leaving the door of refrigerators/refrigerated chambers empty.

The items for conducts with coolers were: organizing thermal boxes for everyday use; conditioning vaccines in coolers with reusable coils and digital thermometers with extension cables; placing the sensor of the thermometer at the center of the thermal box to monitor the temperature until it reaches a minimum of +1° C; placing vaccines in coolers after ambiance and sealing them with a duct tape to clean the refrigerator; transferring vaccines from refrigerators to coolers after 30 minutes until the internal space of the box is between +2° C and +8° C (+5° C is ideal); placing vaccines at the center of the box inside plastic containers for better organization and identification; fitting in recyclable coils before they are placed in coolers; placing reusable coils (0° C) and arranging them in the internal part of coolers; changing coils whenever necessary to keep the internal temperature of the box; cleaning and returning coils to freeze after using them; keeping thermal boxes away from direct sunlight and other sources of heat; cleaning thermal boxes by the end of the work shift.

The last item, measures adopted with vaccines under suspicion, had the following topics: distrusting vaccines with any temperatures different from +2°C and +8°C; suspending the use of the vaccines exposed to temperatures different from the recommended; notifying higher instances about changes of temperature when vaccines are received; maintaining vaccines under suspicion at a temperature of +2 ° C to +8 ° C until second orders by higher instances; filling in the form on immunobiological suspicion and sending it to the hierarchically superior body.

After this stage, in the process of evaluation of the protocol by the judges to validate the content and appearance of the items, all of them obtained agreement within the established level (CVC≥0.78). In the first stage, it was possible to achieve a concordance index for the eight analyzed items. Objectivity/desirability were the most evaluated ones (CVC=0.92), followed by relevance/pertinence, which reached a level of agreement of 0.89. Regarding the general estimate, the instrument had CVC=0.87 at this stage.

This overall CVC value demonstrates the reliability of materials. However, the literature recommends that at least two Delphi stages are needed for the validation stage. Seen that, the second stage was carried out, in which the clarity item reached an agreement index of CVC=0.92. On the other hand, the relevance/pertinence, precision and amplitude items had their values decreased (CVC=0.79), in relation to the first stage of Delphi, as well as other items that also had a small variation, which may be related to the loss of participants (Table 1). Although some values showed reduction in the second stage of Delphi, the overall CVC (0.83) remained well evaluated in the second stage.

Table 1
Consensus between the judges in the stages of Delphi 1 and 2 for the items evaluated in the protocol

	Stage Delphi 1			Stage Delphi 2
Items evaluated	Inappropriate n(%)	Partially Suitable / Suitable n(%)	CVC	Inappropriate n(%)	Partially Suitable / Suitable n(%)	CVC
Behavior	1(8.3)	11(91.7)	0.86	1(12.5)	7(87.5)	0.83
Objectivity/desirability	−(−)	12(100.0)	0.92	1(12.5)	7(87.5)	0.83
Simplicity	1(8.3)	11(91.7)	0.86	−(−)	8(100.0)	0.88
Clarity	−(−)	12(100.0)	0.86	−(−)	8(100.0)	0.92
Relevance/pertinence	−(−)	12(100.0)	0.89	2(25.0)	6(75.0)	0.79
Accuracy	1(8.3)	11(91.7)	0.86	1(12.5)	7(87.5)	0.79
Typicality	1(8.3)	11(91.7)	0.86	−(−)	8(100.0)	0.83
Amplitude	1(8.3)	11(91.7)	0.83	1(12.5)	7(87.5)	0.79

• CVC - Content Validity Coefficient

Based on the decisions of judges, and despite small changes in some values in Delphi 2 (when compared to Delphi 1) there was consensus regarding the validity of the protocol among participants, given it assesses what is proposed and has the appropriate content to evaluate what is intended. This aspect can be identified because 75% of the judges recommend the application of the instrument in the practice of primary care services in the second stage, when compared to 58.3% in the first.

Discussion

The construction and validation of a protocol for the evaluation of safe nursing care with vaccines in PHC was developed with methodological rigor to enable scientific knowledge to be accessible to Nursing professionals working in these spaces.

It is understood that protocol validation studies are adopted by Nursing to verify the quality of the instruments developed and are an essential aspect for the credibility and legitimacy of these resources. As a way of accomplishing the validation stage, the Delphi technique is a strategy that aims not only to reach high consensus among a group of experts on the subject matter of the material, but to obtain quality opinions by these participants.^(16,14)

According to the results, as early as in the first stage, the protocol could be considered validated, since the consensus among the judges for all the separated items and of the protocol in its totality were reached, which confers its high reliability. However, even in this context, the second Delphi stage was carried out, since the literature recommends that at least two stages of opinions be consolidated in the validation process.⁽¹⁶⁾

In the validation of instruments, the decrease in the number of participants along the Delphi stages can happen. Therefore, the literature recommends that, in the validation of content and appearance of an instrument, a minimum of six judges is needed to guarantee the development of research without compromising the quality and validity of the material.^(15,17)

In the two Delphi stages and from the data referring to the characterization of judges, it is possible to affirm that Nursing is a category mostly represented by the female genre, evidence that accompanies this profession throughout history, from its origin to its professionalization today. According to research data collected in 2013 by the Oswaldo Cruz Foundation and the Federal Nursing Council, the scenario in the country is formed by 85.1% of women working in this health field.^(18,19)

In its historical evolution, Nursing has generated new knowledge, which contributes to strengthen its value and support within science, from the production and dissemination of articles published in national and international journals. This amount of studies conducted was possible thanks to the expansion and visibility of Nursing postgraduate courses. Such programs favor the development of Science, Technology and Innovation, with impact on the production of knowledge and qualification of the practice.^(20,21)

Therefore, these postgraduate programs seek the quality of research to achieve excellence in the materials produced. Facing this reality, in the field of Health, the elaboration and validation of instruments, like the protocols, is increasing. The items are judged by the collaboration of a group of experienced judges in the area, who evaluate if the content fits what is proposed.⁽²²⁾

From the initial protocol, the judges made recommendations on all items and, for the vaccination room, such proposals resulted in the inclusion of some elements directed to the climatization of the environment, the form to control the temperature of the room, the routine verification of immunizations and the validity of supplies, existence of refrigerators and/or refrigerated chambers and specification about the infectious residues of Group A1.

The Manual for Cold Chains directs that the sites destined to the reception, preparation and distribution of immunobiologicals, such as vaccination rooms, should be protected from the direct incidence of sunlight and have an ambient air temperature between +18° C and +20° C. Likewise, for patient safety patient in regard to vaccines, these places must be refrigerated with air conditioning.^(23,24)

In addition to these aspects, the validity of vaccines must be checked, with particular attention to the appearance of the solution, package condition and the batch number.⁽²⁵⁾ Therefore, during the work routine, the vaccinator must also consider the control of the time limit of use of the stock of supplies, such as the syringes and needles to be used.

Nonetheless, a study carried out with ten vaccination rooms from three different municipalities of Bahia showed that, out of the sites surveyed, only four had an ideal ambient temperature; and in none of these places the validity period of syringes and needles was observed.⁽²⁶⁾

In addition to the inclusion of the item referring to the supplies, the judges suggested replacing the term “domestic refrigerator” by “refrigerated chamber”, which led to its partial inclusion, since it is believed that many vaccination rooms still use only refrigerators, hence the two terms adopted for the instrument: refrigerator/refrigerated chamber.

The use of domestic refrigerators is not recommended for the storage of vaccines because they do not meet the quality criteria and require additional safety measures for organization and temperature control. Therefore, instances that still use these appliances should, in the shortest possible time, replace them by refrigerated chambers.⁽²³⁾

Besides adequate refrigeration equipment, a correct disposal of waste resulting from vaccination activities must be sustained, such as Group A1 infectants, containing live or attenuated microorganisms, including expired expiration date, vacuums or leftover vaccines, and needles and syringes used.⁽²⁵⁾

Some aspects requested for the vaccination room were not included in Delphi 2 because they were directed to matters regarding location, access and identification of this place in the health unit. These suggestions were not suitable for the purpose of this study.

It is noted that for an appropriate refrigerator care the sealing rubber must be tested, in which it is necessary to make sure, after opening the door, that the appliance has been properly closed. This care should be a daily routine, especially at the end of the day, to ensure the conservation of vaccines.^(23,27)

Another aspect suggested by the judges refers to the best days and shifts for defrosting refrigerators. Despite that, this item was not included in the protocol because the literature does not establish days for equipment cleaning. The guidelines are that this procedure should not be performed at the end of the work shift or on the eve of an extended holiday.⁽²³⁾

The process of cleaning and defrosting refrigerators favors the maintenance of their internal temperature and the ideal conditions for vaccines. Therefore, in the item controlling and recording the temperature of refrigerators, an evaluator proposed the verification of three temperature registers in vaccination rooms that work in the third work shift. The question was not included, as it is believed that this is not a reality for most health facilities in the country. Vaccination rooms usually work in two shifts, morning and afternoon, with the suspension of service hours at lunch break.^(24,27)

Considering this reality, the interruption between shifts for a few hours may hamper the access of population and contribute to the occurrence of Lost Opportunity for Vaccination (LOV), considered a critical point in care.⁽²⁴⁾ Thus, it is pertinent that nursing conducts in the vaccination room contemplate aspects that directly impact the adherence to this practice, including conveying information to the population.

Guidance by professionals should be directed to those responsible and/or vaccinated, regarding the immunobiologicals administered and the scheduling of the next vaccines. Consonant to the clarifications, the request for the immunization records is an important action to favor patient safety with vaccines.⁽²⁵⁾

These professionals are also in charge of performing adequate procedures when preparing these products, with special care for the aspiration technique. Recommendations for the multi-dose vial are to perform needle exchange after each dose withdrawal and new aspiration, with drilling of the rubber at different locations, whereby the central part of the cap should be avoided.^(25,28)

In clinical practice, needle exchange for aspiration and application of medications such as vaccines has some advantages, such as: reducing skin or subcutaneous tissue irritation, and the risk of needle contamination; preserving the patient from multiple applications by needle obstruction, with less pain during the application, among others.⁽²⁸⁾

The use of improper practices in the vaccination room may influence the risk of infection, transmitted by needle contamination in dose preparation or vaccine handling, with a higher risk of exposure to an Adverse Post-Vaccination Event (APVE) due to immunization errors – considered a medication error –, and contamination in vaccination. Fragilities in these practices and occurrence of APVEs increased in recent years, which may be a reflection of deficiencies in nursing conducts during activities.⁽¹¹⁾

In the same way that it needs care during preparation, professionals who work in the vaccination room must include in their routine the “right nine” checklist – right medicine, right dose, right shot, right time, right patient, right record, right action, the right way, and right response before administering the vaccine to patients. These steps are important because the administration of drugs is a complex process and legally constitutes an attribution of Nursing, which must be developed in a safe way.^(29–31)

A topic related to the routine to search for defaulters was also suggested. We opted for non-inclusion because this aspect is not directly related to the study on Patient Safety regarding vaccines.

In addition to these aspects, the disposition of vaccines inside the refrigerator is an essential item to be considered, with components suggested by the judges and introduction of some suggestions in the protocol. For this, at the beginning of the daily work, it is important to look for the date and time identification of opening for multidose vials, so as not to exceed the term of its use.^(25,26)

The way vaccines are stored on shelves directly influences the quality of these products. For domestic refrigerators, the organization should be by type (viral or bacterial) and arranged on the second and third shelves. The first shelf and bottom drawer compartment should not contain vaccines.^(23.25)

For refrigerated chambers, there is no need for immunobiologicals to be differentiated by compartment or type, once the temperature inside equipment is evenly distributed. Although ministerial manuals advocate such conduct, inadequacies in the internal organization of refrigerators may compromise the quality of the vaccines offered to the population. Successive failures in these processes may have even contributed to the re-emergence of already controlled vaccine-preventable diseases.^(24,25)

During the organization of vaccines inside refrigerators, the diluents are attached to the corresponding lyophiles (powder), so that during the operation of rooms, the vaccines and their respective diluents are withdrawn from the refrigerators in an enough quantity for the consumption from the spontaneous demand and the vaccination sessions scheduled for each day.⁽²⁵⁾

The item referring to the use of perforated trays was not inserted in the protocol; containers (perforated or not) may be used for the organization of the vaccines.⁽²⁵⁾

Vaccines to be used in the daily routine need to be properly stored, which requires conducts with thermal boxes by professionals performing immunization activities. This item received suggestions by the judges, but they were not included in the protocol.

The analysis made by the participants in both stages was essential to obtain the validation of the protocol of safe nursing care with vaccines regarding its content and appearance. The modifications suggested by the experts in the first Delphi stage contributed to perfect the protocol, which resulted in the positive evaluation and, consequently, in its recommendation by the judges for the application of the instrument in the practice of the health services under the PHC.

For this study, the agreement between the items evaluated by the experts, with CVC scores >0.78 for all the separated items and the protocol in general, as recommended,⁽¹⁵⁾ represents a relevant aspect in the process of validating the protocol, which guarantees its high credibility. As a next step of the study, the importance of applying the protocol in the primary care vaccination rooms is emphasized.

The limitations of research were the scarcity of materials on the construction and validation of technologies for the safety of patient regarding vaccines, as well as the reduction in sample size in the second Delphi stage.

Conclusion

The safety protocol for vaccine care was elaborated and validated by the judges and reached agreement on all individual items and the instrument itself, according to the established requirements. The use of this protocol is considered relevant in the country, in the sense of collaborating with interventions and safe actions, within the spectrum of overcoming the difficulties experienced in the implementation of immunization practices. Its adoption in health institutions can contribute to the quality of nursing care regarding vaccines and perfect the conduct of professionals. It is believed that further research on this subject is essential to improve knowledge and ensure qualified and safe care in Nursing regarding vaccines.

Referências

• ¹
  Thofehrn MB, Montesinos MJ, Arrieira IC, Àvila VC, Vasques TC, Farias ID. Processo de trabalho dos enfermeiros de um hospital da espanha: ênfase nas tecnologias de cuidado. Cogitare Enferm. 2014;19(1):141–6.
• ²
  Oliveira RM, Leitão IM, Silva LM, Figueiredo SV, Sampaio RL, Gondim MM. Estratégias para promover segurança do paciente: da identificação dos riscos às práticas baseadas em evidências. Esc Anna Nery. 2014;18(1):122–9.
• ³
  Paes GO, Mello EC, Leite JL, Mesquita MG, Oliveira FT, Carvalho SM. Protocolo de cuidados ao cliente com distúrbio respiratório: ferramenta para tomada de decisão aplicada à enfermagem. Esc Anna Nery. 2014;18(2):303–10.
• ⁴
  Catunda HL, Bernardo EB, Vasconcelos CT, Moura ER, Pinheiro AK, Aquino PS. Percurso metodológico em pesquisas de enfermagem para construção e validação de protocolos. Texto Contexto Enferm. 2017;26(2):1–10.
• ⁵
  Mourão Netto JJ, Dias MA, Goyanna NF. Uso de instrumentos enquanto tecnologia para a saúde. Saúde Redes. 2016;2(1):65–72.
• ⁶
  Vieira AC, Bertoncello KC, Girondi JB, Nascimento ER, Hammerschmidt KS, Zeferinho MT. Percepção dos enfermeiros de emergência na utilização de um protocolo para avaliação da dor torácica. Texto Contexto Enferm. 2016;25(1):1–7.
• ⁷
  Salvador PT, Rodrigues CC, Lima KY, Alves KY, Santos VE. Tecnologias não convencionais de coleta de dados em dissertações e teses brasileiras. Rev Bras Enferm. 2015;68(2):269–77.
• ⁸
  Veras JE, Joventino ES, Coutinho JF, Lima FE, Rodrigues AP, Ximenes LB. Classificação de risco em pediatria: construção e validação de um guia para enfermeiros. Rev Bras Enferm. 2015;68(5):913-22.
• ⁹
  Silva AS, Gubert FA, Lima ICV, Rolim RM, Tavares DR, Silva DA, et al. Validação de conteúdo e aparência de um curso online para a vigilância da influenza. Rev Ibero-Am Estudos Educ. 2017; 12(Esp. 2):1408-20.
• ¹⁰
  Interaminense IN, Oliveira SC, Leal LP, Linhares FM, Pontes CM. Tecnologias educativas para promoção da vacinação contra o papilomavírus humano: revisão integrativa da literatura. Texto Contexto Enferm. 2016;25(2):1–10.
• ¹¹
  Bisetto LH, Ciosak SI. Análise da ocorrência de evento adverso pós-vacinação decorrente de erro de imunização. Rev Bras Enferm. 2017;70(1):87–95.
• ¹²
  Oliveira VC, Gallardo MP, Arcêncio RA, Gontijo TL, Pinto IC. Avaliação da qualidade de conservação de vacinas na atenção Primária à saúde. Cien Saude Colet. 2014;19(9):3889–98.
• ¹³
  Fehring RJ. The Fehring Model. In: M. Paquete R. Carrol-Johnson. Classification of nursing diagnoses: proceedings of the tenth conference of North American Nursing Diagnosis Association. Philadelphia: Lippincott; 1994.
• ¹⁴
  Revorêdo LS, Maia RS, Torres GV, Chaves Maia EM. O uso da técnica Delphi em saúde: uma revisão integrativa de estudos brasileiros. Arq Ciênc Saúde. 2015;22(2):16–21.
• ¹⁵
  Pasquali L. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010.
• ¹⁶
  Rozados HB. O uso da técnica Delphi como alternativa metodológica para a área da Ciência da Informação. Em Questão. 2015; 21(3):64-86.
• ¹⁷
  Pereira RD, Alvim NA. Técnica Delphi no diálogo com enfermeiros sobre acupuntura como proposta de intervenção de enfermagem. Esc Anna Nery. 2015;19(1):174–80.
• ¹⁸
  Souza LL, Araújo DB, Silva DS, Bêrredo VC. Representações de gênero na prática de enfermagem na perspectiva de estudantes. Ciênc Cogn. 2014;19(2):218–32.
• ¹⁹
  Machado MH, Aguiar Filho W, Lacerda WF, Oliveira E, Lemos W, Wermelinger M, et al. Características gerais da enfermagem: o perfil sócio demográfico. Enferm Foco. 2016; 7(Esp.):9-14.
• ²⁰
  Scochi CG, Munari DB, Gelbcke FL, Ferreira MA. Desafios e estratégias dos programas de pós-graduação em enfermagem para a difusão da produção científica em periódicos internacionais. Esc Anna Nery. 2014;18(1):5-6.
• ²¹
  Marziale MH, Lima RA. A formação de doutores e a produção do conhecimento em enfermagem. Rev Lat Am Enfermagem. 2015;23(3):361–2.
• ²²
  Medeiros RK, Ferreira Júnior MA, Pinto DP, Vitor AF, Santos VE, Barichello E. Modelo de validação de conteúdo de Pasquali nas pesquisas em Enfermagem. Rev Enferm Ref. 2015;4(4):127–35.
• ²³
  Brasil. Ministério da Saúde. Manual de rede de frio. 4a ed. Brasília (DF): Ministério da Saúde; 2013.
• ²⁴
  Deus SE, Marques AD, Teixeira JC, Deus PR, Moraes ME, Macêdo DS. Estudo dos procedimentos quanto à conservação das vacinas do Programa Nacional de Imunização. Rev Enferm UFPE Online. 2016;10(3):1038–46.
• ²⁵
  Brasil. Ministério da Saúde. Manual de normas e procedimentos para vacinação. Brasília (DF): Ministério da Saúde, 2014. 176 p.
• ²⁶
  Rios NM, Carneiro BF, Andrade RM. Aspectos observacionais das salas de vacina de três municípios do interior da Bahia. Rev Inov Tecnol Ciênc. 2015;1(1):63–70.
• ²⁷
  Siqueira LD, Martins AM, Versiani CM, Almeida LA, Oliveira CD, Nascimento JE, et al. Avaliação da organização e funcionamento das salas de vacina na Atenção Primária à Saúde em Montes Claros, Minas Gerais, 2015. Epidemiol Serv Saude. 2017;26(3):557–68.
• ²⁸
  Conselho Regional de Enfermagem (COREN). Ementa: Troca de agulha para administração de medicamento por via intramuscular. Parecer COREN-SP 010/2009, atualizado em 21/09/2015. São Paulo: COREN; 2015.
• ²⁹
  Conselho Regional de Enfermagem (COREN). Uso seguro de medicamentos: guia para preparo, administração e monitoramento / Conselho Regional de Enfermagem de São Paulo. São Paulo: COREN; 2017.
• ³⁰
  Bohomol E. Erros de medicação: estudo descritivo das classes dos medicamentos e medicamentos de alta vigilância. Esc Anna Nery. 2014;18(2):311–6.
• ³¹
  Pereira CD, Tourinho FS, Santos VE. Segurança do paciente: avaliação do sistema de medicação por enfermeiros utilizando análise fotográfica. Enferm Foco. 2016;7(1):76–80.

Appendix 1 References used as a rationale for elaborating the protocol

References Ahmed M, Arona S, McKay J, Long S, Vincent C, Kelly M, et al. Patient safety skills in primary care: a national survey of GP educators. BMC Fam Pract. 2014;15(206):1-8. Alameddine M, Saleh S, Natafgi N. Assessing health-care providers' readiness for reporting quality and patient safety indicators at primary health-care centres in Lebanon: a national cross-sectional survey. Hum Resour Health. 2015;13(37):1-14. Aranda CMSS, Moraes JC. Cold net for the conservation of vaccines in public units of the city of São Paulo: knowledge and practice. Rev bras epidemiol. 2006;9(2):172-8. Astier-Peña MP, Torijano-Casalengua ML, Olivera-Cañadas G, Silvestre-Busto C, Agra-Varela Y, Maderuelo-Fernández JA. Are Spanish primary care professionals aware of patient safety? Eur J Public Health. 2015;25(5):781–7. Bodur S, Filiz E. A survey on patient safety culture in primary healthcare services in Turkey. Int J Qual Health Care. 2009;21(5):348-55. Bondevik GT, Hofoss D, Hansen EH, Deilkâs ECT. Patient safety culture in Norwegian primary care: A study in out-of-hours casualty clinics and GP practices. Scand J Prim Health Care. 2014a;32(3):132-8. Bondevik GT, Hofoss D, Hansen EH, Deilkâs ECT. The safety attitudes questionnaire – ambulatory version: psychometric properties of the Norwegian translated version for the primary care setting. BMC Health Serv Res. 2014b;14(139):1-10. Brazil. Cold Net Manual. Brasília, DF: Ministério da Saúde, 2013. Brazil. Manual of Norms and Procedures for vaccination. Brasília, DF: Ministério da Saúde, 2014. Cabral IC. Evaluation of the Local Cold Net of the Sanitary District III of João Pessoa -B [dissertation]. João Pessoa: Universidade Federal da Paraíba; 2013. Car LT, Papachristou N, Gallagher J, Samra R, Wazny K, El-Khatib M, et al. Identification of priorities for improvement of medication safety in primary care: a prioritize study. BMC Fam Pract. 2016;17(1):1-10. Carr C, Byles J, Durrheim D. Practice nurses best protect the vaccine cold chain in general practice. Aust J Adv Nurs. 2009;27(2):35-9. Crane S, Sloane PD, Elder N, Cohen L, Laughtenschlaeger N, Walsh K, et al. Reporting and Using Near-miss Events to Improve Patient Safety in Diverse Primary Care Practices: A Collaborative Approach to Learning from Our Mistakes. J Am Board Fam Med. 2015;28(4):452-60. Daker-White G, Hays R, McSharry J, Giles S, Cheraghi-Sohi S, Rhodes P, et al. Blame the Patient, Blame the Doctor or Blame the System? A Meta-Synthesis of Qualitative Studies of Patient Safety in Primary Care. PLoS One. 2015;10(8):1-42. Douglas H, Pace WD, Harris, DM, West DR, Main DS, Westfall JM. Event Reporting to a Primary Care Patient Safety Reporting System: A Report From the ASIPS Collaborative. Ann Fam Med. 2004;2(4):1-6. Dowell D, Manwell LB, Maguire A, An PG, Paluch L, Felix K, et al. Urban Outpatient Views on Quality and Safety in Primary Care. Healthc Q. 2005;3(1):2-8. Ely JW, Kaldjian LC, D'Alessandro DM. Diagnostic Errors in Primary Care: Lessons Learned. J Am Board Fam Med. 2012;25(1):87-97. Fiandt K, Doeschot C, Lanning J. Characteristics of risk in patients of nurse practitioner safety net practices. J Am Acad Nurse Pract. 2010;22(9):474-9. Gaal S, Hombergh PVD, Verstappen W, Wensing M. Patient safety features are more present in larger primary care practices. Health Policy. 2010;97(1):87-91. Gaal S, Verstappen W, Wensing M. Patient safety in primary care: a survey of general practitioners in the Netherlands. BMC Health Serv Res. 2010;10(21):1-7. Gaal S, Verstappen W, Wensing M. What do primary care physicians and researchers consider the most important patient safety improvement strategies? BMC Health Serv Res. 2011;11(102):1-6. Gabrani JC, Knibb W, Petrela E, Hoxha A, Gabrani A. Provider Perspectives on Safety in Primary Care in Albania. J Nurs Scholarsh. 2016;48(6):552-60. Gehring K, Schwappach DLB, Battaglia M, Buff R, Huber F, Sauter P, et al. Safety climate and its association with office type and team involvement in primary care. Int J Qual Health Care. 2013;25(4):394-402. Gehring K, Schwappach DLB, Battaglia M, Buff R, Huber F, Sauter P, et al. Frequency of and Harm Associated With Primary Care Safety Incidents. Am J Manag Care. 2012;18(9):323-37. Grant AM, Guthrie B, Dreischulte T. Developing a complex intervention to improve prescribing safety in primary care: mixed methods feasibility and optimisation pilot study. BMJ Open. 2014;4(1):1-8. Hernan AL, Giles SJ, Fuller J, Johnson JK, Walker C, Dunbar JA. Patient and carer identified factors which contribute to safety incidents in primary care: a qualitative study. BMJ Qual Saf. 2015;24(9):583-93. Kostopoulou O; Delaney B. Confidential reporting of patient safety events in primary care: results from a multilevel classification of cognitive and system factors. Qual Saf Health Care. 2007;16(2):95-100. Luna GLM, Vieira LJES, Souza PF, Lira SVG, Moreira DP, Pereira AS. Aspects related to the administration and conservation of vaccines in health centers in Northeast Brazil. Ciênc Saúde Colet. 2011;16(2):513-21. Mahfouz AA, Abdelmoneim I, Khan MY, Daffalla AA, Diab MM, Shaban H, et al. Injection safety at primary health care level in south-western Saudi Arabia. East Mediterr Health J. 2009;15(2):443-50. Marchon SG, Mendes Junior WV. Patient safety in primary health care: systematic review. Cad Saude Publica. 2014;30(9):1-21. Marchon SG, Mendes Junior WV, Pavão ALB. Characteristics of adverse events in primary health care in Brazil. Cad Saude Publica. 2015;31(11):2313-30. McGuire MJ, Noronha G, Samal L, Yeh HC, Crocetti S, Kravet S. Patient Safety Perceptions of Primary Care Providers after Implementation of an Electronic Medical Record System. J Gen Intern Med. 2013;28(2):184-92. Mclaws ML, Ghahramani S, Palenik CJ, Keshtkar V, Askarian M. Assessment of injection practice in primary health care facilities of Shiraz, Iran. Am J Infect Control. 2014;42(3):1-5. Mira JJ, Nebot C, Lorenzo S, Pérez-Jover V. Patient report on information given, consultation time and safety in primary care. Qual Saf Health Care. 2010;19(5):1-4. Oliveira IP. Production of nursing team care in immunization services in family health care [dissertation]. Feira de Santana: State University of Feira de Santana; 2014. Oliveira VC, Gallardo MDPS, Arcêncio RA, Gontijo TL, Pinto IC. Evaluation of conservation quality of vaccines in primary health care. Cien e Saude Colet. 2014;19(9):3889-98. Oliveira VC, Guimarães EAA, Guimarães IA, Januário LH, Pinto IC. Nursing practice in the conservation of vaccines. Acta Paul Enferm. 2009;22(6):814-8. Oliveira VC. Evaluation of conservation quality of vaccines in Primary Health Care Units of the Central-West Region of Minas Gerais. Ribeirão Preto. Thesis [PhD in Health Sciences] - University of São Paulo; 2012. Panagioti M, Stokes J, Esmail A, Coventry P, Cheraghi-Sohi S, Alam R, et al. Multimorbidity and Patient Safety Incidents in Primary Care: A Systematic Review and Meta-Analysis. PLoS One. 2015;10(8):1-30. Paranaguá TTB, Bezerra ALQ, Tobias GC, Ciosak SI. Support for learning from the perspective of patient safety in primary health care. Rev Lat Am Enfermagem. 2016;24:1-9. Rees P, Edwards A, Panesar S, Powell C, Carter B, Williams H, et al. Safety Incidents in the Primary Care Office Setting. Pediatrics. 2015;135(6):1-9. Rhodes P, McDonald R, Campbell S, Daker-White G, Sanders C. Sensemaking and the co-production of safety: a qualitative study of primary medical care patients. Sociol Health e Illn. 2016;38(2):270-85. Rhodes P, Campbell S, Sanders C. Trust, temporality and systems: how do patients understand patient safety in primary care? A qualitative study. Health Expect. 2016;19(2):253-63. Samra R, Bottle A, Aylin P. Monitoring patient safety in primary care: an exploratory study using in-depth semistructured interviews. BMJ Open. 2015;5(9):1-7. Sánchez MR, Borrell-Carrió F, Parra CO, Fernàndez N, Gallego DAF. Auditings in clinical safety for primary care centers. Pilot study. Primary Care. 2013;45(7):341-8. Schwappach DLB, Gehring K, Battaglia M, Buff R, Huber F, Sauter P, et al. Threats to patient safety in the primary care office: concerns of physicians and nurses. Swiss Med Wkly. 2012;142:1-10. Shi L, Lebrun-Harris LA, Parasuraman SR, Zhu J, Ngo-Metzger Q. The Quality of Primary Care Experienced by Health Center Patients. J Am Board Fam Med. 2013;26(6):768-77. Smith-Strøm H, Thornes I. Patient safety regarding leg ulcer treatment in primary health care. Nord J Nurs Res. 2008;28(3):29-33. Smits M, Huibers L, Kerssemeijer B, Feijter E, Wensing M, Giesen P. Patient safety in out-of-hours primary care: a review of patient records. BMC Health Serv Res. 2010;10(335):1-9. Tabrizchi N, Sedaghat M. The First Study of Patient Safety Culture in Iranian Primary Health Centers. Acta Medica Iranica. 2012;50(7):505-10. Torijano-Casalengua ML, Olivera-Cañadas G, Astier-Peña MP, Maderuelo-Fernández JÁ, Silvestre-Busto C. Validation of a questionnaire to evaluate the patient safety culture of primary care professionals in Spain. Primary School. 2013;45(1):21-37. Tsang C, Bottle A, Majeed A, Aylin P. Adverse events recorded in English primary care: observational study using the General Practice Research Database. Br J Gen Pract. 2013;63(613):534-42. Verbakel NJ, Melle MV, Langelaan M, Verheij TJM, Wagner C, Zwart DLM. Exploring patient safety culture in primary care. Int J Qual Health Care. 2014;26(6):585-91. Verstappen W, Gaal S, Bowie P, Parker D, Lainer M, Valderas JM, et al. A research agenda on patient safety in primary care. Recommendations by the LINNEAUS collaboration on patient safety in primary care. Eur J Gen Pract. 2015;21(1):72-7. Verstappen W, Gaal S, Esmail A, Wensing M. Patient safety improvement programmes for primary care. Review of a Delphi procedure and pilot studies by the LINNEAUS collaboration on patient safety in primary care. Eur J Gen Pract. 2015;21(1):50-5. Wallis K, Dovey S. No-fault compensation for treatment injury in New Zealand: identifying threats to patient safety in primary care. BMJ Qual Saf. 2011;20(7):587-91. Wammes JJG, Verstappen W, Gaal S, Wensing M. Organisational targets of patient safety improvement programs in primary care; an international web-based survey. BMC Fam Pract. 2013;14(145):1-7. Webair HH, Al-assani SS, Al-haddad RH, Al-Shaeeb WH, Selm MAB, Alyamani AS. Assessment of patient safety culture in primary care setting, Al-Mukala, Yemen. BMC Fam Pract. 2015;16(136):1-9. Zwart DLM, Bont AA. Introducing incident reporting in primary care: a translation from safety science into medical practice. Health, Risk & Society. 2013;15(3):265-78.

References
Ahmed M, Arona S, McKay J, Long S, Vincent C, Kelly M, et al. Patient safety skills in primary care: a national survey of GP educators. BMC Fam Pract. 2014;15(206):1-8.
Alameddine M, Saleh S, Natafgi N. Assessing health-care providers' readiness for reporting quality and patient safety indicators at primary health-care centres in Lebanon: a national cross-sectional survey. Hum Resour Health. 2015;13(37):1-14.
Aranda CMSS, Moraes JC. Cold net for the conservation of vaccines in public units of the city of São Paulo: knowledge and practice. Rev bras epidemiol. 2006;9(2):172-8.
Astier-Peña MP, Torijano-Casalengua ML, Olivera-Cañadas G, Silvestre-Busto C, Agra-Varela Y, Maderuelo-Fernández JA. Are Spanish primary care professionals aware of patient safety? Eur J Public Health. 2015;25(5):781–7.
Bodur S, Filiz E. A survey on patient safety culture in primary healthcare services in Turkey. Int J Qual Health Care. 2009;21(5):348-55.
Bondevik GT, Hofoss D, Hansen EH, Deilkâs ECT. Patient safety culture in Norwegian primary care: A study in out-of-hours casualty clinics and GP practices. Scand J Prim Health Care. 2014a;32(3):132-8.
Bondevik GT, Hofoss D, Hansen EH, Deilkâs ECT. The safety attitudes questionnaire – ambulatory version: psychometric properties of the Norwegian translated version for the primary care setting. BMC Health Serv Res. 2014b;14(139):1-10.
Brazil. Cold Net Manual. Brasília, DF: Ministério da Saúde, 2013.
Brazil. Manual of Norms and Procedures for vaccination. Brasília, DF: Ministério da Saúde, 2014.
Cabral IC. Evaluation of the Local Cold Net of the Sanitary District III of João Pessoa -B [dissertation]. João Pessoa: Universidade Federal da Paraíba; 2013.
Car LT, Papachristou N, Gallagher J, Samra R, Wazny K, El-Khatib M, et al. Identification of priorities for improvement of medication safety in primary care: a prioritize study. BMC Fam Pract. 2016;17(1):1-10.
Carr C, Byles J, Durrheim D. Practice nurses best protect the vaccine cold chain in general practice. Aust J Adv Nurs. 2009;27(2):35-9.
Crane S, Sloane PD, Elder N, Cohen L, Laughtenschlaeger N, Walsh K, et al. Reporting and Using Near-miss Events to Improve Patient Safety in Diverse Primary Care Practices: A Collaborative Approach to Learning from Our Mistakes. J Am Board Fam Med. 2015;28(4):452-60.
Daker-White G, Hays R, McSharry J, Giles S, Cheraghi-Sohi S, Rhodes P, et al. Blame the Patient, Blame the Doctor or Blame the System? A Meta-Synthesis of Qualitative Studies of Patient Safety in Primary Care. PLoS One. 2015;10(8):1-42.
Douglas H, Pace WD, Harris, DM, West DR, Main DS, Westfall JM. Event Reporting to a Primary Care Patient Safety Reporting System: A Report From the ASIPS Collaborative. Ann Fam Med. 2004;2(4):1-6.
Dowell D, Manwell LB, Maguire A, An PG, Paluch L, Felix K, et al. Urban Outpatient Views on Quality and Safety in Primary Care. Healthc Q. 2005;3(1):2-8.
Ely JW, Kaldjian LC, D'Alessandro DM. Diagnostic Errors in Primary Care: Lessons Learned. J Am Board Fam Med. 2012;25(1):87-97.
Fiandt K, Doeschot C, Lanning J. Characteristics of risk in patients of nurse practitioner safety net practices. J Am Acad Nurse Pract. 2010;22(9):474-9.
Gaal S, Hombergh PVD, Verstappen W, Wensing M. Patient safety features are more present in larger primary care practices. Health Policy. 2010;97(1):87-91.
Gaal S, Verstappen W, Wensing M. Patient safety in primary care: a survey of general practitioners in the Netherlands. BMC Health Serv Res. 2010;10(21):1-7.
Gaal S, Verstappen W, Wensing M. What do primary care physicians and researchers consider the most important patient safety improvement strategies? BMC Health Serv Res. 2011;11(102):1-6.
Gabrani JC, Knibb W, Petrela E, Hoxha A, Gabrani A. Provider Perspectives on Safety in Primary Care in Albania. J Nurs Scholarsh. 2016;48(6):552-60.
Gehring K, Schwappach DLB, Battaglia M, Buff R, Huber F, Sauter P, et al. Safety climate and its association with office type and team involvement in primary care. Int J Qual Health Care. 2013;25(4):394-402.
Gehring K, Schwappach DLB, Battaglia M, Buff R, Huber F, Sauter P, et al. Frequency of and Harm Associated With Primary Care Safety Incidents. Am J Manag Care. 2012;18(9):323-37.
Grant AM, Guthrie B, Dreischulte T. Developing a complex intervention to improve prescribing safety in primary care: mixed methods feasibility and optimisation pilot study. BMJ Open. 2014;4(1):1-8.
Hernan AL, Giles SJ, Fuller J, Johnson JK, Walker C, Dunbar JA. Patient and carer identified factors which contribute to safety incidents in primary care: a qualitative study. BMJ Qual Saf. 2015;24(9):583-93.
Kostopoulou O; Delaney B. Confidential reporting of patient safety events in primary care: results from a multilevel classification of cognitive and system factors. Qual Saf Health Care. 2007;16(2):95-100.
Luna GLM, Vieira LJES, Souza PF, Lira SVG, Moreira DP, Pereira AS. Aspects related to the administration and conservation of vaccines in health centers in Northeast Brazil. Ciênc Saúde Colet. 2011;16(2):513-21.
Mahfouz AA, Abdelmoneim I, Khan MY, Daffalla AA, Diab MM, Shaban H, et al. Injection safety at primary health care level in south-western Saudi Arabia. East Mediterr Health J. 2009;15(2):443-50.
Marchon SG, Mendes Junior WV. Patient safety in primary health care: systematic review. Cad Saude Publica. 2014;30(9):1-21.
Marchon SG, Mendes Junior WV, Pavão ALB. Characteristics of adverse events in primary health care in Brazil. Cad Saude Publica. 2015;31(11):2313-30.
McGuire MJ, Noronha G, Samal L, Yeh HC, Crocetti S, Kravet S. Patient Safety Perceptions of Primary Care Providers after Implementation of an Electronic Medical Record System. J Gen Intern Med. 2013;28(2):184-92.
Mclaws ML, Ghahramani S, Palenik CJ, Keshtkar V, Askarian M. Assessment of injection practice in primary health care facilities of Shiraz, Iran. Am J Infect Control. 2014;42(3):1-5.
Mira JJ, Nebot C, Lorenzo S, Pérez-Jover V. Patient report on information given, consultation time and safety in primary care. Qual Saf Health Care. 2010;19(5):1-4.
Oliveira IP. Production of nursing team care in immunization services in family health care [dissertation]. Feira de Santana: State University of Feira de Santana; 2014.
Oliveira VC, Gallardo MDPS, Arcêncio RA, Gontijo TL, Pinto IC. Evaluation of conservation quality of vaccines in primary health care. Cien e Saude Colet. 2014;19(9):3889-98.
Oliveira VC, Guimarães EAA, Guimarães IA, Januário LH, Pinto IC. Nursing practice in the conservation of vaccines. Acta Paul Enferm. 2009;22(6):814-8.
Oliveira VC. Evaluation of conservation quality of vaccines in Primary Health Care Units of the Central-West Region of Minas Gerais. Ribeirão Preto. Thesis [PhD in Health Sciences] - University of São Paulo; 2012.
Panagioti M, Stokes J, Esmail A, Coventry P, Cheraghi-Sohi S, Alam R, et al. Multimorbidity and Patient Safety Incidents in Primary Care: A Systematic Review and Meta-Analysis. PLoS One. 2015;10(8):1-30.
Paranaguá TTB, Bezerra ALQ, Tobias GC, Ciosak SI. Support for learning from the perspective of patient safety in primary health care. Rev Lat Am Enfermagem. 2016;24:1-9.
Rees P, Edwards A, Panesar S, Powell C, Carter B, Williams H, et al. Safety Incidents in the Primary Care Office Setting. Pediatrics. 2015;135(6):1-9.
Rhodes P, McDonald R, Campbell S, Daker-White G, Sanders C. Sensemaking and the co-production of safety: a qualitative study of primary medical care patients. Sociol Health e Illn. 2016;38(2):270-85.
Rhodes P, Campbell S, Sanders C. Trust, temporality and systems: how do patients understand patient safety in primary care? A qualitative study. Health Expect. 2016;19(2):253-63.
Samra R, Bottle A, Aylin P. Monitoring patient safety in primary care: an exploratory study using in-depth semistructured interviews. BMJ Open. 2015;5(9):1-7.
Sánchez MR, Borrell-Carrió F, Parra CO, Fernàndez N, Gallego DAF. Auditings in clinical safety for primary care centers. Pilot study. Primary Care. 2013;45(7):341-8.
Schwappach DLB, Gehring K, Battaglia M, Buff R, Huber F, Sauter P, et al. Threats to patient safety in the primary care office: concerns of physicians and nurses. Swiss Med Wkly. 2012;142:1-10.
Shi L, Lebrun-Harris LA, Parasuraman SR, Zhu J, Ngo-Metzger Q. The Quality of Primary Care Experienced by Health Center Patients. J Am Board Fam Med. 2013;26(6):768-77.
Smith-Strøm H, Thornes I. Patient safety regarding leg ulcer treatment in primary health care. Nord J Nurs Res. 2008;28(3):29-33.
Smits M, Huibers L, Kerssemeijer B, Feijter E, Wensing M, Giesen P. Patient safety in out-of-hours primary care: a review of patient records. BMC Health Serv Res. 2010;10(335):1-9.
Tabrizchi N, Sedaghat M. The First Study of Patient Safety Culture in Iranian Primary Health Centers. Acta Medica Iranica. 2012;50(7):505-10.
Torijano-Casalengua ML, Olivera-Cañadas G, Astier-Peña MP, Maderuelo-Fernández JÁ, Silvestre-Busto C. Validation of a questionnaire to evaluate the patient safety culture of primary care professionals in Spain. Primary School. 2013;45(1):21-37.
Tsang C, Bottle A, Majeed A, Aylin P. Adverse events recorded in English primary care: observational study using the General Practice Research Database. Br J Gen Pract. 2013;63(613):534-42.
Verbakel NJ, Melle MV, Langelaan M, Verheij TJM, Wagner C, Zwart DLM. Exploring patient safety culture in primary care. Int J Qual Health Care. 2014;26(6):585-91.
Verstappen W, Gaal S, Bowie P, Parker D, Lainer M, Valderas JM, et al. A research agenda on patient safety in primary care. Recommendations by the LINNEAUS collaboration on patient safety in primary care. Eur J Gen Pract. 2015;21(1):72-7.
Verstappen W, Gaal S, Esmail A, Wensing M. Patient safety improvement programmes for primary care. Review of a Delphi procedure and pilot studies by the LINNEAUS collaboration on patient safety in primary care. Eur J Gen Pract. 2015;21(1):50-5.
Wallis K, Dovey S. No-fault compensation for treatment injury in New Zealand: identifying threats to patient safety in primary care. BMJ Qual Saf. 2011;20(7):587-91.
Wammes JJG, Verstappen W, Gaal S, Wensing M. Organisational targets of patient safety improvement programs in primary care; an international web-based survey. BMC Fam Pract. 2013;14(145):1-7.
Webair HH, Al-assani SS, Al-haddad RH, Al-Shaeeb WH, Selm MAB, Alyamani AS. Assessment of patient safety culture in primary care setting, Al-Mukala, Yemen. BMC Fam Pract. 2015;16(136):1-9.
Zwart DLM, Bont AA. Introducing incident reporting in primary care: a translation from safety science into medical practice. Health, Risk & Society. 2013;15(3):265-78.

Appendix 2 Protocol for evaluating the safety of nursing care regarding vaccines during Primary Care

The appearance of the protocol had as reference the proposal of Domansky,⁽¹⁾ with the use of symbols standardized at international level. These elements, when applied, allow to describe and / or establish the sequence of a process, in which each figure has a specific meaning.

Thus, the ellipse - an oval figure - indicates the beginning and end of the process. The diamond - referring to the seven items listed in check-list -, serves in making decisions. The rectangle refers to the actions to be implemented, the arrow guides the process direction, the full line guides the path, and the dashed line connects a conduct to an explanatory box.

From these aspects, for the application of the protocol, measurement criteria were established. Therefore, for each category evaluated, a score was adopted: Suitable (2 points); Partially Appropriate (1 point); Inappropriate (0 points).

With this, a protocol category will be adequate if all check-list items are present at the time of observation. It will be partially adequate if more than half of the evaluated items are contemplated. Inappropriateness will be attributed to items that have less than half of the topics.

In light of this, after application of the protocol, the categories will be counted at from the assigned score. The sum will allow to evaluate if the nursing performance regarding vaccines favors a safe, partially safe care or insecure for the patient.

• 1
  Domansky RC. Elaboration of protocol. In: Domansky RC, Borges EL. (orgs.). Manual for prevention of skin lesions: recommendations based on evidence. 2. Ed. Rio de Janeiro: Rubio; 2014. p. 231-272.

Publication Dates

• Publication in this collection
  Jan-Feb 2019

History

• Received
  23 Aug 2018
• Accepted
  22 Jan 2019