Acessibilidade / Reportar erro

Analysis of immunization errors in pregnant women

Análisis de los errores de inmunización en gestantes

ABSTRACT

Objective:

To analyze immunization errors in pregnant women from Minas Gerais, according to the absence and presence of Adverse Events Following Immunization.

Methods:

This is a cross-sectional study, carried out with data on immunization errors in pregnant women, between 2015 and 2019, registered in the Information System for the Surveillance of Adverse Events, in Minas Gerais. The trend of the incidence rate of immunization error per 100,000 doses applied was checked using Prais-Winsten models.

Results:

Of all notifications, 3.72% were vaccination errors with adverse events following immunization. The highest proportion of immunization errors with no adverse events (32.40%) was in the macro-region Centro and, with adverse events (27.78%), in Triângulo do Sul, both with a stationary trend in the period. Regarding the incidence rate, the macro-region with the highest immunization errors with no adverse events was Vale do Aço and the macro-region with the highest incidence rate of immunization errors with adverse events was Triângulo do Sul.

Conclusion:

In this study, the notifications of Adverse Events Following Immunization resulting from vaccination errors with and without adverse events occurred with no significant trend within the years of the study.

DESCRIPTORS
Vaccination; Immunization; Drug-Related Side Effects and Adverse Reactions; Medication Errors; Pregnant Women; Prenatal Care

RESUMEN

Objetivo:

Analizar los errores de inmunización en gestantes de Minas Gerais, Brasil, según ausencia y presencia de Evento Adverso Postvacunal.

Métodos:

Estudio transversal, realizado con datos de errores de inmunización en gestantes, entre los años 2015 y 2019, registrados en el Sistema de Información de Vigilancia de Eventos Adversos, en Minas Gerais. La tendencia de la tasa de incidencia de error de inmunización por 100 mil dosis utilizadas fue verificada por los modelos de Prais-Winsten.

Resultados:

3,72% de todas las notificaciones fueron errores de inmunización con evento adverso postvacunal. La gran proporción de errores de inmunización sin evento adverso (32,40%) fue en la macrorregión Centro y, con evento adverso (27,78%), en la Triângulo do Sul, ambos con tendencia de estabilización en el período. En lo que se refiere a la tasa de incidencia, la macrorregión con mayores errores de inmunización sin evento adverso fue la región de Vale do Aço y la macrorregión con mayor tasa de incidencia de errores de inmunización con evento adverso fue la Triângulo do Sul.

Conclusión:

En este estudio, las notificaciones de Evento Adverso Post vacuna provenientes de errores de inmunización sin y con eventos adversos ocurrieron sin tendencia significativa entre los años de estudio.

DESCRIPTORES
Vacunación; Inmunización; Efectos Colaterales y Reacciones Adversas Relacionados con Medicamentos; Errores de Medicación; Mujeres Embarazadas; Atención Prenatal

RESUMO

Objetivo:

Analisar os erros de imunização em gestantes mineiras, segundo ausência e presença de Evento Adverso Pós-Vacinação.

Métodos:

Estudo transversal, realizado com dados de erros de imunização em gestantes, entre os anos 2015 e 2019, registrados no Sistema de Informação da Vigilância de Eventos Adversos, em Minas Gerais. A tendência da taxa de incidência de erro de imunização por 100 mil doses aplicadas foi verificada pelos modelos de Prais-Winsten.

Resultados:

De todas as notificações, 3,72% foram erros de imunização com evento adverso pós-vacinação. A maior proporção de erros de imunização sem evento adverso (32,40%) foi na macrorregião Centro e, com evento adverso (27,78%), na Triângulo do Sul, ambos com tendência estacionária no período. Em relação à taxa de incidência, a macrorregião com maiores erros de imunização sem evento adverso foi a Vale do Aço e a macrorregião com maior taxa de incidência de erros de imunização com evento adverso foi a Triângulo do Sul.

Conclusão:

Neste estudo, as notificações de Evento Adverso Pós-Vacinação decorrentes de erros de imunização sem e com eventos adversos ocorreram sem tendência significativa entre os anos do estudo.

DESCRITORES
Vacinação; Imunização; Efeitos Colaterais e Reações Adversas Relacionados a Medicamentos; Erros de Medicação; Gestantes; Cuidado Pré-Natal

INTRODUCTION

Immunization is a great achievement in public health in human history(11. World Health Organization. Vaccine Safety Basics e-learning course. Module 1: introduction to vaccine safety basics [Internet]. Geneva: WHO; c2020 [cited 2020 Mar 28]. Avaliable from: https://vaccine-safety-training.org/overview-and-outcomes-1.html.
https://vaccine-safety-training.org/over...
), as it helps in the prevention, elimination, and eradication of vaccine-preventable diseases(22. Bisetto LHL, Ciosak SI. Análise da ocorrência de evento adverso pós-vacinação decorrente de erro de imunização. Rev Bras Enferm. 2017;70(1):87-95. DOI: https://doi.org/10.1590/0034-7167-2016-0034.
https://doi.org/10.1590/0034-7167-2016-0...
33. Brasil. Ministério da Saúde [Internet]. Brasília; 2017 [cited 2020 Mar 30]. Avaliable from: https://www.gov.br/saude/pt-br/acesso-a-informacao/acoes-e-programas/programa-nacional-de-imunizacoes-vacinacao.
https://www.gov.br/saude/pt-br/acesso-a-...
). To control these diseases in Brazil, vaccines began to be used in the 19th century. In the early 1970s, the National Immunization Program (PNI) was created to organize the national policy on vaccination of the Brazilian population and to be in charge of the surveillance of immunization actions in Brazil(44. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Programa nacional de Imunizações: 40 anos [Internet]. 1ᵃ ed. Brasília; 2013 [cited 2020 Mar 30]. Avaliable from: https://bvsms.saude.gov.br/bvs/publicacoes/programa_nacional_imunizacoes_pni40.pdf
https://bvsms.saude.gov.br/bvs/publicaco...
).

It is known that the expected effect of vaccines is that, under ideal conditions, they protect the population against the diseases or infections for which they were produced. However, immunobiologicals are not risk-free(55. World Health Organization. Vaccine Safety Basics – learning manual [Internet]. Geneva: WHO; 2013 [cited 2020 Apr 3]. Avaliable from: https://www.who.int/vaccine_safety/initiative/tech_support/Vaccine-safety-E-course-manual.pdf.
https://www.who.int/vaccine_safety/initi...
). In this context, the Adverse Event Following Immunization (AEFI) refers to an unwanted or untoward event (symptom, disease, or abnormal laboratory finding) following immunization and which may or may not be related to the use of vaccines, immunoglobulins, or heterologous sera(66. World Health Organization. Definition and Application of Terms for Vaccine Pharmacovigilance. Report of CIOMS/WHO Working Group on Vaccine Pharmacovigilance [Internet]. Geneva: WHO; 2012 [cited 2020 Apr 3]. Avaliable from: https://www.who.int/vaccine_safety/initiative/tools/CIOMS_report_WG_vaccine.pdf.
https://www.who.int/vaccine_safety/initi...
). Therefore, a strategy is sought to minimize such AEFIs in care practice(66. World Health Organization. Definition and Application of Terms for Vaccine Pharmacovigilance. Report of CIOMS/WHO Working Group on Vaccine Pharmacovigilance [Internet]. Geneva: WHO; 2012 [cited 2020 Apr 3]. Avaliable from: https://www.who.int/vaccine_safety/initiative/tools/CIOMS_report_WG_vaccine.pdf.
https://www.who.int/vaccine_safety/initi...
).

As the monitoring of immunobiologicals safety shall be rigorous and, to ensure better benefit for the vaccinated population, with greater adherence to vaccination, several countries have created, in line with the recommendations of the World Health Organization (WHO), adverse events following immunization surveillance (AEFIS)(22. Bisetto LHL, Ciosak SI. Análise da ocorrência de evento adverso pós-vacinação decorrente de erro de imunização. Rev Bras Enferm. 2017;70(1):87-95. DOI: https://doi.org/10.1590/0034-7167-2016-0034.
https://doi.org/10.1590/0034-7167-2016-0...
,77. Monteiro SAMG, Takano OA, Waldman EA. Avaliação do sistema brasileiro de vigilância de eventos adversos pós-vacinação. Rev Bras Epidemiol. 2011;14(3):361-71. DOI: https://doi.org/10.1590/S1415-790X2011000300002.
https://doi.org/10.1590/S1415-790X201100...
).

In 1992, PNI started AEFI surveillance through the National AEFI Surveillance System(22. Bisetto LHL, Ciosak SI. Análise da ocorrência de evento adverso pós-vacinação decorrente de erro de imunização. Rev Bras Enferm. 2017;70(1):87-95. DOI: https://doi.org/10.1590/0034-7167-2016-0034.
https://doi.org/10.1590/0034-7167-2016-0...
) and, in 1998, it published the 1st edition of the Manual for the Epidemiological Surveillance of Adverse Events Following Immunization(88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
99. Pacheco FC, Domingues CMAS, Maranhão AGK, Carvalho SMD, Teixeira AMS, Braz RM, et al. Análise do Sistema de Informação da Vigilância de Eventos Adversos Pós-Vacinação no Brasil, 2014 a 2016. Rev Panam Salud Publica. 2018;42:e12. DOI: https://doi.org/10.26633/RPSP.2018.12.
https://doi.org/10.26633/RPSP.2018.12...
). As of 2000, the Information System for the Adverse Events Following Immunization Surveillance (IS-AEFI) started being implemented, but compulsory notification was only instituted in 2005(88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
99. Pacheco FC, Domingues CMAS, Maranhão AGK, Carvalho SMD, Teixeira AMS, Braz RM, et al. Análise do Sistema de Informação da Vigilância de Eventos Adversos Pós-Vacinação no Brasil, 2014 a 2016. Rev Panam Salud Publica. 2018;42:e12. DOI: https://doi.org/10.26633/RPSP.2018.12.
https://doi.org/10.26633/RPSP.2018.12...
).

AEFIs, resulting from immunization errors (IE), have been increasing(55. World Health Organization. Vaccine Safety Basics – learning manual [Internet]. Geneva: WHO; 2013 [cited 2020 Apr 3]. Avaliable from: https://www.who.int/vaccine_safety/initiative/tech_support/Vaccine-safety-E-course-manual.pdf.
https://www.who.int/vaccine_safety/initi...
). They refer to any preventable event resulting from errors in the preparation, handling, storage, or administration(55. World Health Organization. Vaccine Safety Basics – learning manual [Internet]. Geneva: WHO; 2013 [cited 2020 Apr 3]. Avaliable from: https://www.who.int/vaccine_safety/initiative/tech_support/Vaccine-safety-E-course-manual.pdf.
https://www.who.int/vaccine_safety/initi...
,88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
) of immunobiologicals that can reduce or cancel the expected vaccine effect(88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
). They are classified as: production error (non-compliance with good manufacturing practices that can lead to quality escape, such as changes in potency and increase in reactogenicity), cold chain error (vaccine transported/stored incorrectly), handling error, and administration error (non-sterile injection, reconstitution error, injection in the wrong place, contraindication ignored, expired vaccine), which occur due to non-compliance with standards and techniques, resulting or not in a possible adverse event(22. Bisetto LHL, Ciosak SI. Análise da ocorrência de evento adverso pós-vacinação decorrente de erro de imunização. Rev Bras Enferm. 2017;70(1):87-95. DOI: https://doi.org/10.1590/0034-7167-2016-0034.
https://doi.org/10.1590/0034-7167-2016-0...
,55. World Health Organization. Vaccine Safety Basics – learning manual [Internet]. Geneva: WHO; 2013 [cited 2020 Apr 3]. Avaliable from: https://www.who.int/vaccine_safety/initiative/tech_support/Vaccine-safety-E-course-manual.pdf.
https://www.who.int/vaccine_safety/initi...
,88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
).

Besides the potential harm to pregnant women, IE can have a negative impact on the population’s confidence in vaccination, which influences the follow-up of the vaccination schedule and, consequently, the reduction of vaccine coverage, putting the control of vaccine-preventable diseases at risk(88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
). It should be noted that pregnancy is a process marked by immunological and physiological changes in women that contribute to changes in mothers’ and fetuses’ susceptibility to infections, with alterations in the immune response(1010. Alvarez AMR, Jauregui B, El Omeiri N. Progress towards a comprehensive approach to maternal and neonatal immunization in the Americas. Rev Panam Salud Publica. 2017;41:e159. DOI: https://doi.org/10.26633/RPSP.2017.159.
https://doi.org/10.26633/RPSP.2017.159...
). There is also the importance of the pregnant woman’s vaccination for the protection of her child, through the transfer of maternal antibodies(1010. Alvarez AMR, Jauregui B, El Omeiri N. Progress towards a comprehensive approach to maternal and neonatal immunization in the Americas. Rev Panam Salud Publica. 2017;41:e159. DOI: https://doi.org/10.26633/RPSP.2017.159.
https://doi.org/10.26633/RPSP.2017.159...
).

In the light of the above-mentioned, the aim of this study was to analyze IE in pregnant women from Minas Gerais, according to the absence and presence of AEFI. The identification of IE in pregnant women and of the AEFI resulting from them is in the interest of public health. Thus, the findings of this study may collaborate to improve the quality of care management and of the surveillance system, professional training, and to reduce harm to pregnant women and their children.

METHOD

Design of Study

This is a cross-sectional study.

Local

This study was carried out with IE notifications registered in the AEFI database, of the IS-PNI, in the state of Minas Gerais (MG), Brazil, from January 1, 2015 to December 31, 2019.

The state of MG has 853 municipalities, spread over a territory of 586,522.122 km, with a population of 21,168,791 inhabitants in 2019. Due to its demographic, socioeconomic, geographic, health and epidemiological characteristics, MG was divided into 14 macro-regions, considered for health care planning. They are: Centro-Sul, Centro, Jequitinhonha, Oeste, Leste, Sul, Sudeste, Norte, Noroeste, Leste do Sul, Nordeste, Vale do Aço, Triângulo Sul, and Triângulo Norte(1111. Governo do Estado de Minas Gerais. Secretaria de Estado de Saúde. Deliberação CIB-SUS/MG No 3.013, de 23 de outubro de 2019. Aprova o Ajuste/2019 do Plano Diretor de Regionalização PDR/SUSMG e dá outras providências. Diário Oficial do Estado de Minas Gerais [Internet]. Belo Horizonte; 2019 [cited 2020 Mar 27]. Avaliable from: https://www.saude.mg.gov.br/images/documentos/Del%203013%20-%20SUBGR_SDCAR_DREA%20-%20Ajuste%20PDR%20vers%c3%a3o%20CIB%20-%20alterada%2015.10.pdf.
https://www.saude.mg.gov.br/images/docum...
).

The information was taken from the AEFI database (module IS-AEFI) registered at the IS-PNI, in MG.

Selection Criteria

The inclusion criteria consisted of notifications of IE in pregnant women with or without adverse events. The exclusion criteria included AEFI resulting from IE with notification before January 1, 2015 and after December 31, 2019 in the state of MG, which were in the database.

To avoid information bias, only notifications registered from January 1, 2015 were analyzed, since the period of system implementation included July 1, 2014, even if some federative units had carried out the feedback of suspected AEFI cases in the IS-AEFI. This measure was adopted to prevent municipalities that subsequently implemented the IS-AEFI from having low notifications when compared to those that already had the system implemented.

Data Collection

Data collection was carried out in January 2020, with variables selection: year of occurrence of AEFI; health macro-region of occurrence of AEFI; immunobiological administered; IE with AEFI and IE without AEFI; medical care (yes, no, and ignored) and case progression (healing with sequelae, healing without sequelae, in follow-up, it was not AEFI, and no information).

According to the national vaccination schedule recommended for pregnant women(88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
), AEFI notifications were analyzed, and the immunobiologicals were divided into: vaccines recommended during pregnancy – adsorbed diphtheria, tetanus, (acellular) pertussis vaccine (DTaP), adsorbed adult diphtheria and tetanus vaccine (Td), hepatitis B vaccine (recombinant), and influenza; vaccines recommended in special situations – adsorbed hepatitis A vaccine (inactivated), pneumococcal, meningococcal conjugate ACWY/C vaccine, meningococcal B, vero cell rabies vaccine, and yellow fever vaccine; and vaccines contraindicated for pregnant women – in this category, any vaccine other than those recommended during pregnancy or recommended in special situations was considered, as it is understood that, if administered, they constitute errors in administration in pregnant women(88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
).

The IS, according to the absence and presence of adverse events in pregnant women, referring to the immunobiologicals administered, were divided into three categories: recommended during pregnancy, recommended in special situations, and contraindicated.

Data Analysis and Treatment

To calculate the IR of IE per 100,000 doses applied, the number of IE with and without adverse event was considered as numerator and, as the denominator, the number of doses administered to pregnant women by period and health macro-region. The number of doses was obtained from the Ministry of Health (MS) website, at “<pni.datasus.gov.br>”. Only the doses applied to pregnant women when the age group was 10 to 49 years old and the dose was administered during the gestational period were considered.

The statistical package Statistical Software for Professional (Stata), version 14.0, was used for data analysis. Estimates of AEFI resulting from IE were presented in proportions (%) according to: the year of occurrence, the health macro-region of Minas Gerais, and the immunobiological given. Data were stratified according to IE with no adverse event and with adverse event.

To identify the existence of an IR trend of IE per 100,000 applied doses, the Prais-Winsten generalized linear analysis models were used. Prais-Winsten regression model is indicated to correct the serial self-correlation from time series. For the Prais-Winsten regression, the transformation of the IR from IE to the logarithmic scale was performed. This process is performed to reduce the heterogeneity of the variance of the residuals from the regression analysis. The calculation of the average annual percent change (APC) was also performed for each dependent variable analyzed. To calculate the APC, the following formula was used: APC = (−1 + 10 [b1] * 100%), where b1 refers to the slope of Prais-Winsten regression. The 95% confidence intervals (95% CI) of the APC measurements were also calculated, using the following formula: Minimum 95% CI = (−1 + 10 [b1 − t * e] * 100%); and maximum 95% CI = (−1 + 10 [b1 + t * e] * 100%), in which the values of the coefficient b1 and (standard error) were generated by the statistical analysis software; t refers to t-student and corresponds to 4 degrees of freedom(1212. Antunes JLF, Cardoso MRA. Uso da análise de séries temporais em estudos epidemiológicos. Epidemiol. Serv. Saúde. 2015;24(3): 565-576. DOI: http://dx.doi.org/10.5123/S1679-49742015000300024.
http://dx.doi.org/10.5123/S1679-49742015...
). The regression results were interpreted as follows: increasing trend when p value was lower than 0.05 and regression coefficient was positive), decreasing trend when p value was lower than 0.05 and negative regression coefficient) or stationary trend when p value was higher than 0.05(1212. Antunes JLF, Cardoso MRA. Uso da análise de séries temporais em estudos epidemiológicos. Epidemiol. Serv. Saúde. 2015;24(3): 565-576. DOI: http://dx.doi.org/10.5123/S1679-49742015000300024.
http://dx.doi.org/10.5123/S1679-49742015...
).

Ethical Aspects

The research on “Vaccination of pregnant women: evaluation of epidemiological and clinical aspects in the city of Belo Horizonte” was approved by the Ethics Committee of the Universidade Federal de Minas Gerais, under opinion number 1.507.859 on April 19, 2016.

RESULTS

In the period from 2015 to 2019, 484 notifications of immunization errors in pregnant women were reported in the state of MG. Most IE did not result in adverse events (96.28%). However, 3.72% were IE with an adverse event (Table 1). The highest proportions of IE without (27.78%) and with (42.70%) adverse events occurred in 2017 and the lowest rates in 2016 (5.36%) (no adverse event) and in 2015 (11.11%) (with adverse event) (Table 1).

Table 1 -
Proportion, incidence rate and trend analysis of the Immunization Error according to absence and presence of adverse event following immunization in pregnant women – Minas Gerais, Brazil, 2015–2019.

Regarding IR, it is observed that AEFI resulting from IE with no adverse events had a rate of 53.49 cases for every 100,000 doses administered in pregnant women during the period. The year 2017 had the highest IR of AEFI arising from IE for both of them (with and without AEFI). It should be noted that in the period both IRs showed a stable trend over the years (p = 0.207; p = 0.496, respectively) (Table 1).

The macro-region Centro, which has the largest number of pregnant women in the state (32.87% of the total pregnant population in MG), recorded the highest proportion of IE with no adverse events (32.40%) in the state. The macro-region Triângulo do Sul, in its turn, had the highest proportion of IE with an adverse event (27.78%). The macroregion with the lowest proportion of IE with no adverse events was Jequitinhonha (Table 2). Regarding IR, the macro-region with the highest IE with no AEFI was Vale do Aço, where 105 AEFI were reported, equivalent to an IR of 233.87 cases of IE with no AEFI for every 100,000 doses administered to pregnant women. The macro-region with the highest IR of IE with adverse event was Triângulo do Sul, with 5 cases and IR of 16.50 cases of IE with AEFI for every 100,000 doses administered (Table 2).

Table 2 -
Ratio and Incidence Rate of the Immunization Error according to absence and presence of adverse event following immunization in pregnant women, in health macro-regions – Minas Gerais, Brazil, 2015–2019.

The highest proportion of IE with no adverse events (41.17%) corresponded to vaccines recommended in special situations, followed by 34.24% of contraindicated vaccines. However, 24.59% of IE with no adverse events were due to the recommended vaccines during the gestational period. Regarding vaccines involved with adverse events resulting from IE, the MMR recorded a higher percentage of adverse events (35%) – this being contraindicated during pregnancy, followed by Yellow Fever (30%) – recommended only in special situations.

Regarding the vaccines recommended for pregnant women, the Triple Bacterial Acellular (DTaP) vaccine was the one with the most IE with no adverse events (10.75%), followed by Hepatitis B (6.38%), adult combination (Td) (4 .92%) and Influenza (2.55%). Among the vaccines recommended in special situations, the Yellow Fever vaccine was responsible for 99.12% of all IE with no adverse events in this category. Regarding vaccines contraindicated during pregnancy, the MMR vaccine presented 88.83% of IE with no adverse event in this category. It should be noted that the same woman may have taken more than one immunobiological at a time (Table 3).

Table 3 -
Immunization error according to absence and presence of adverse event following immunization in pregnant women, according to the recommendations of the Ministry of Health of Brazil for the gestational and immunobiological period. Minas Gerais, Brazil, 2015–2019.

Medical care took place in 0.21% and 44.44% of IE cases with and without adverse events, respectively, totaling 1.86% of all reported IE. Of the total of 18 notifications of IE according to the presence of adverse event following immunization, 16.67% were healed with no sequelae and 5.56% were healed with sequelae. Among all cases of IE, 1.65% were followed up (Table 4).

Table 4 -
Immunization error according to absence and presence of adverse event following immunization in pregnant women, according to medical care and case progression – Minas Gerais, Brazil, 2015–2019.

DISCUSSION

In this study, the notifications of AEFI resulting from IE without adverse events occurred with no significant trend within the years of the study. The IE with adverse events in pregnant women showed, in their turn, an increase until the year 2017, with stabilization in the following years. However, the IR in the trend analysis was stationary during the study period.

The highest rates of IE with and without adverse events occurred in 2017, and authors suggest different factors that could potentially favor an increase in the number of adverse events: improved surveillance of AEFI by municipalities; deficiencies in nursing vaccination practice and changes in the National Vaccination Schedule(22. Bisetto LHL, Ciosak SI. Análise da ocorrência de evento adverso pós-vacinação decorrente de erro de imunização. Rev Bras Enferm. 2017;70(1):87-95. DOI: https://doi.org/10.1590/0034-7167-2016-0034.
https://doi.org/10.1590/0034-7167-2016-0...
,1313. Sociedade Brasileira de Imunizações. Calendário de Vacinação SBIm Gestante 2020/2021 [Internet]. São Paulo; 2020 [cited 2020 May 5]. Avaliable from: https://sbim.org.br/images/calendarios/calend-sbim-gestante.pdf.
https://sbim.org.br/images/calendarios/c...
). In 2017, one of the changes that may have had a direct impact on pregnant women was the reduction in the gestational age for the administration of DTaP (which was between the 27th and 36th weeks and changed to the 20th week)(1313. Sociedade Brasileira de Imunizações. Calendário de Vacinação SBIm Gestante 2020/2021 [Internet]. São Paulo; 2020 [cited 2020 May 5]. Avaliable from: https://sbim.org.br/images/calendarios/calend-sbim-gestante.pdf.
https://sbim.org.br/images/calendarios/c...
,1414. Brasil. Ministério da Saúde. Blog da saúde [Internet]. Brasília; 2017 Mar [cited 2020 May 5]. Avaliable from: http://www.blog.saude.gov.br/index.php/promocao-da-saude/52392-veja-o-que-muda-no-calendario-de-vacinacao-em-2017.
http://www.blog.saude.gov.br/index.php/p...
,1515. Secretaria de Estado de Saúde de Minas Gerais [Internet]. 2017 [cited 2020 May 5]. Avaliable from: https://www.saude.mg.gov.br/aedes/page/1607-campanha-nacional-de-multivacinacao-2017.
https://www.saude.mg.gov.br/aedes/page/1...
).

On the other hand, the lowest rates of IE were observed in 2016 with no adverse events, with a discrepancy when compared to the others, and in 2015 with adverse events. Such results can be related to the low adherence to the registration of notifications – which may be linked to the recent implementation of the IS-AEFI online module, in 2014. This requires adaptation, training, and adequate knowledge of professionals to register notifications(88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
99. Pacheco FC, Domingues CMAS, Maranhão AGK, Carvalho SMD, Teixeira AMS, Braz RM, et al. Análise do Sistema de Informação da Vigilância de Eventos Adversos Pós-Vacinação no Brasil, 2014 a 2016. Rev Panam Salud Publica. 2018;42:e12. DOI: https://doi.org/10.26633/RPSP.2018.12.
https://doi.org/10.26633/RPSP.2018.12...
).

The variation in AEFI notifications due to IE registered in the macro-regions, presented in this study, can be explained by the implementation of the online system that did not occur simultaneously in all macro-regions, as well as the use of computers in the vaccine room – also boosted in 2014(1616. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Informática do SUS. Sistema de Informação do Programa Nacional de Imunizações SIPNI: Manual do usuário do SIPNI (Desktop) [Internet]. Brasília; 2018 [cited 2020 May 8]. Avaliable from: http://pni.datasus.gov.br/sipni/documentos/manual_sipni.pdf.
http://pni.datasus.gov.br/sipni/document...
). Authors also reveal other factors, such as the lack of information about the importance of reporting AEFI(99. Pacheco FC, Domingues CMAS, Maranhão AGK, Carvalho SMD, Teixeira AMS, Braz RM, et al. Análise do Sistema de Informação da Vigilância de Eventos Adversos Pós-Vacinação no Brasil, 2014 a 2016. Rev Panam Salud Publica. 2018;42:e12. DOI: https://doi.org/10.26633/RPSP.2018.12.
https://doi.org/10.26633/RPSP.2018.12...
) and structural problems (such as the slow use of the system and the short time available to fill in the notification forms), which can interfere with the registration of suspected cases of AEFI(99. Pacheco FC, Domingues CMAS, Maranhão AGK, Carvalho SMD, Teixeira AMS, Braz RM, et al. Análise do Sistema de Informação da Vigilância de Eventos Adversos Pós-Vacinação no Brasil, 2014 a 2016. Rev Panam Salud Publica. 2018;42:e12. DOI: https://doi.org/10.26633/RPSP.2018.12.
https://doi.org/10.26633/RPSP.2018.12...
).

In this study, it was observed that all macro-regions had IE with no adverse events, with the highest proportion in the macro-region Centro, an expected result – since this is the most populous macro-region with the highest vaccination coverage(1717. Santos G, Pales RC. Estratégias e desenvolvimento em Minas Gerais: uma análise comparada das macrorregiões de planejamento. Cadernos do Desenvolvimento [Internet]. 2014 [cited 2020 May 8];9(14):163-185. Avaliable from: http://www.cadernosdodesenvolvimento.org.br/ojs-2.4.8/index.php/cdes/article/view/129/132.
http://www.cadernosdodesenvolvimento.org...
1818. Costa NMN, Leão AMM. Casos notificados de eventos adversos pós-vacinação: contribuição para o cuidar em enfermagem. Rev Enferm UERJ [Internet]. 2015 [cited 2020 May 15];23(3):297-303. DOI: https://doi.org/10.12957/reuerj.2015.14850.
https://doi.org/10.12957/reuerj.2015.148...
). This finding corroborates another study, carried out nationwide, in which the highest incidences were in regions of greater development(99. Pacheco FC, Domingues CMAS, Maranhão AGK, Carvalho SMD, Teixeira AMS, Braz RM, et al. Análise do Sistema de Informação da Vigilância de Eventos Adversos Pós-Vacinação no Brasil, 2014 a 2016. Rev Panam Salud Publica. 2018;42:e12. DOI: https://doi.org/10.26633/RPSP.2018.12.
https://doi.org/10.26633/RPSP.2018.12...
), such as the macro-region Centro of the State.

However, the highest proportion of IE with adverse events was found in the macro-region Triângulo do Sul, which may be due to underreporting in other macro-regions – for example, there are Jequitinhonha, Leste, Leste do Sul, Triângulo do Norte, Sudeste and Centro-sul, which did not register any cases. This finding corroborates a study carried out in Paraná that identified underreporting of AEFI, in which some municipalities had no records(22. Bisetto LHL, Ciosak SI. Análise da ocorrência de evento adverso pós-vacinação decorrente de erro de imunização. Rev Bras Enferm. 2017;70(1):87-95. DOI: https://doi.org/10.1590/0034-7167-2016-0034.
https://doi.org/10.1590/0034-7167-2016-0...
). Therefore, improvement in the training of technicians is required, as well as the search for new strategies to improve epidemiological surveillance(22. Bisetto LHL, Ciosak SI. Análise da ocorrência de evento adverso pós-vacinação decorrente de erro de imunização. Rev Bras Enferm. 2017;70(1):87-95. DOI: https://doi.org/10.1590/0034-7167-2016-0034.
https://doi.org/10.1590/0034-7167-2016-0...
).

Among the categories of immunobiologicals studied, vaccines recommended in special situations during pregnancy, followed by contraindicated vaccines, represented the highest ratios of IE records with no adverse events. Regarding IE with the presence of adverse events, this was different, since contraindicated vaccines present the highest ratio. Taking the immunobiologicals themselves into account, yellow fever and MMR vaccines accounted for the greatest increase in this number. It should be noted that the state of Minas Gerais, in 2017 and 2018, tackled an epidemiological outbreak of yellow fever, which may justify the increase in immunization errors for the vaccine in question(1919. Secretaria de Estado de Saúde de Minas Gerais. NOTA TÉCNICA CONJUNTA – DVE/SVEAST/DPAPS/CSPPL/SAPS/SES-MG Nº 04/2018 [cited 2020 Apr 10]. Avaliable from: https://www.saude.mg.gov.br/images/noticias_e_eventos/000_2018/BoletinsEpidemiologicos/NOTA%20TCNICA%20FA%2004_2018%20FINAL.pdf.
https://www.saude.mg.gov.br/images/notic...
).

A study carried out in the state of Goiás, with data from the IS-PNI/IS-AEFI, confirms these findings, since it revealed a higher ratio of IE related to the vaccine against measles, mumps, rubella (attenuated) (MMR) (15,4%) and yellow fever (12.0%)(2020. Barboza TC, Guimarães RA, Gimenes FRE, Silva AEBC. Retrospective study of immunization errors reported in an online Information System. Rev Lat Am Enfermagem. 2020;28:e3303. DOI: https://doi.org/10.1590/1518-8345.3343.3303.
https://doi.org/10.1590/1518-8345.3343.3...
) too. This can be explained by failures in the prescription and/or indication of the immunobiological, type of immunobiological used, and error in the evaluation of contraindications or prescriptions(2020. Barboza TC, Guimarães RA, Gimenes FRE, Silva AEBC. Retrospective study of immunization errors reported in an online Information System. Rev Lat Am Enfermagem. 2020;28:e3303. DOI: https://doi.org/10.1590/1518-8345.3343.3303.
https://doi.org/10.1590/1518-8345.3343.3...
). The importance of a multidisciplinary team with up-to-date knowledge about the recommended vaccination schedule for this public is highlighted, especially that of the prenatal nurse, to change this scenario.

Another finding of the study was related to the category, vaccines recommended during the gestational period, which also accounted for IE records, but in a smaller proportion. In cases of IE with no adverse events, the adsorbed diphtheria, tetanus, (acellular) pertussis (DTaP) vaccine was the most common, and in cases of IE with adverse events, DTaP and dual adult – Td had the same number of records. The homogenization of immunobiologicals is essential when they contain an adjuvant(2121. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de Normas e Procedimentos para Vacinação [Internet]. 1ᵃ ed. Brasília; 2014 [cited 2020 May 20]. Avaliable from: http://bvsms.saude.gov.br/bvs/publicacoes/manual_procedimentos_vacinacao.pdf.
http://bvsms.saude.gov.br/bvs/publicacoe...
), as is the case with the aforementioned vaccines, which use aluminum hydroxide as a substance to stimulate immune system responses(88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
,1818. Costa NMN, Leão AMM. Casos notificados de eventos adversos pós-vacinação: contribuição para o cuidar em enfermagem. Rev Enferm UERJ [Internet]. 2015 [cited 2020 May 15];23(3):297-303. DOI: https://doi.org/10.12957/reuerj.2015.14850.
https://doi.org/10.12957/reuerj.2015.148...
). Thus, not performing such a procedure can be considered an IE, with the possible occurrence of an adverse event.

A survey conducted in Europe with the analysis of IE obtained incomplete vaccination (36.1%), administration errors (22.1%), and wrong vaccine or age (14.6%) as the most frequent results(2222. Hoeve CE, Haren A, Sturkenboom MCJM, Straus SMJM. Spontaneous reports of vaccination errors in the European regulatory database EudraVigilance: a descriptive study. Vaccine. 2018;36(52):7956-64. DOI: https://doi.org/10.1016/j.vaccine.2018.11.003
https://doi.org/10.1016/j.vaccine.2018.1...
). In the United States, a study indicated that the most reported IE were inappropriate regimen (27%), storage errors, dispensing of immunobiologicals (23.0%), and incorrect vaccine (15.0%)(2323. Hibbs BF, Moro PL, Lewis P, Miller ER, Shimabukuro TT. Vaccination errors reported to the Vaccine Adverse Event Reporting System, (VAERS) United States, 2000-2013. Vaccine. 2015;33(28):3171-8. DOI: https://doi.org/10.1016/j.vaccine.2015.05.006.
https://doi.org/10.1016/j.vaccine.2015.0...
). Other studies pointed to errors in the administration stage as the most common(2424. Rees P, Edwards A, Powell C, Evans HP, Carter B, Hibbert P, et al. Pediatric immunization-related safety incidents in primary care: a mixed methods analysis of a national database. Vaccine. 2015;33(32):3873-80. DOI: https://doi.org/10.1016/j.vaccine.2015.06.068.
https://doi.org/10.1016/j.vaccine.2015.0...
2525. Institute for Safe Medication Practices [Internet]. c2018 [cited 2020 June 8]. Avaliable from: https://www.ismp.org/resources/ismp-national-vaccine-errors-reporting-program-2017-analysis-part-i-vaccine-errors.
https://www.ismp.org/resources/ismp-nati...
), including wrong number of doses, inadequate interval between doses, and incorrect vaccine(2525. Institute for Safe Medication Practices [Internet]. c2018 [cited 2020 June 8]. Avaliable from: https://www.ismp.org/resources/ismp-national-vaccine-errors-reporting-program-2017-analysis-part-i-vaccine-errors.
https://www.ismp.org/resources/ismp-nati...
). It should be noted that severe AEFI resulting from IE are rare(2626. World Health Organization. Global Advisory Committee on Vaccine Safety. Safety of Immunization during Pregnancy: a review of the evidence [Internet]. Geneva: WHO; 2014 [citado 2020 June 11]. Avaliable from: https://www.who.int/vaccine_safety/publications/safety_pregnancy_nov2014.pdf?ua=1.
https://www.who.int/vaccine_safety/publi...
) and that the benefits of immunizing pregnant women are significantly greater and outweigh the possible potential risks(2626. World Health Organization. Global Advisory Committee on Vaccine Safety. Safety of Immunization during Pregnancy: a review of the evidence [Internet]. Geneva: WHO; 2014 [citado 2020 June 11]. Avaliable from: https://www.who.int/vaccine_safety/publications/safety_pregnancy_nov2014.pdf?ua=1.
https://www.who.int/vaccine_safety/publi...
). Therefore, prenatal consultations are an important moment between the pregnant woman and the health service, especially for health education regarding vaccination(1919. Secretaria de Estado de Saúde de Minas Gerais. NOTA TÉCNICA CONJUNTA – DVE/SVEAST/DPAPS/CSPPL/SAPS/SES-MG Nº 04/2018 [cited 2020 Apr 10]. Avaliable from: https://www.saude.mg.gov.br/images/noticias_e_eventos/000_2018/BoletinsEpidemiologicos/NOTA%20TCNICA%20FA%2004_2018%20FINAL.pdf.
https://www.saude.mg.gov.br/images/notic...
,2626. World Health Organization. Global Advisory Committee on Vaccine Safety. Safety of Immunization during Pregnancy: a review of the evidence [Internet]. Geneva: WHO; 2014 [citado 2020 June 11]. Avaliable from: https://www.who.int/vaccine_safety/publications/safety_pregnancy_nov2014.pdf?ua=1.
https://www.who.int/vaccine_safety/publi...
).

Regarding AEFI clinical progression, the results of this research showed that, of the notifications of IE with adverse events, only one case was healed with sequelae. This is confirmed by findings in the literature, which demonstrate that the risks of adverse events are outweighed by the benefits of using vaccines in pregnant women to prevent infectious diseases that can affect the fetus(2727. Silva SS, Oliveira VC, Ribeiro HCTC, Alves TGS, Cavalcante RB, Guimarães EAA. Análise dos eventos adversos após aplicação de vacinas em Minas Gerais, 2011: um estudo transversal. Epidemiol Serv Saúde. 2016;25(1):45-54. DOI: http://dx.doi.org/10.5123/s1679-49742016000100005.
http://dx.doi.org/10.5123/s1679-49742016...
2828. Moro PL, Zheteyeva Y, Barash F, Lewis P, Cano M. Assessing the safety of hepatitis B vaccination during pregnancy in the Vaccine Adverse Event Reporting System (VAERS), 1990–2016. Vaccine. 2018;36(1):50-4. DOI: https://doi.org/10.1016/j.vaccine.2017.11.039.
https://doi.org/10.1016/j.vaccine.2017.1...
). A study also carried out in the state of Minas Gerais showed that the most adopted approach was to maintain the vaccination schedule, since, in most cases, adverse events are benign and transient(2727. Silva SS, Oliveira VC, Ribeiro HCTC, Alves TGS, Cavalcante RB, Guimarães EAA. Análise dos eventos adversos após aplicação de vacinas em Minas Gerais, 2011: um estudo transversal. Epidemiol Serv Saúde. 2016;25(1):45-54. DOI: http://dx.doi.org/10.5123/s1679-49742016000100005.
http://dx.doi.org/10.5123/s1679-49742016...
). However, most cases of IE with adverse events do not have complete information, which can impact the quality of the information provided.

Another result of this study showed that, of the IE records according to the absence and presence of adverse events, most of them had medical care ignored. Moreover, there is the result related to the case progression – in which a small part was followed up. It is also noteworthy that most of the events notified were terminated with no information on the case progression, which leads to infer that there were errors in the notifications filling(99. Pacheco FC, Domingues CMAS, Maranhão AGK, Carvalho SMD, Teixeira AMS, Braz RM, et al. Análise do Sistema de Informação da Vigilância de Eventos Adversos Pós-Vacinação no Brasil, 2014 a 2016. Rev Panam Salud Publica. 2018;42:e12. DOI: https://doi.org/10.26633/RPSP.2018.12.
https://doi.org/10.26633/RPSP.2018.12...
).

The inadequate filling of the fields hampers the identification of other factors that may be related to the occurrence of AEFI resulting from IE in pregnant women. This can impair proper conduct in the face of cases, since such practice can generate data favoring vaccination discredit(2828. Moro PL, Zheteyeva Y, Barash F, Lewis P, Cano M. Assessing the safety of hepatitis B vaccination during pregnancy in the Vaccine Adverse Event Reporting System (VAERS), 1990–2016. Vaccine. 2018;36(1):50-4. DOI: https://doi.org/10.1016/j.vaccine.2017.11.039.
https://doi.org/10.1016/j.vaccine.2017.1...
). As a result, the population and the professionals themselves can relate vaccines to events that constitute a temporal association(99. Pacheco FC, Domingues CMAS, Maranhão AGK, Carvalho SMD, Teixeira AMS, Braz RM, et al. Análise do Sistema de Informação da Vigilância de Eventos Adversos Pós-Vacinação no Brasil, 2014 a 2016. Rev Panam Salud Publica. 2018;42:e12. DOI: https://doi.org/10.26633/RPSP.2018.12.
https://doi.org/10.26633/RPSP.2018.12...
), negatively impacting vaccination coverage(88. Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. Manual de vigilância epidemiológica de eventos adversos pós-vacinação [Internet]. 4ᵃ ed. Brasília; 2020 [cited 2020 Apr 29]. Avaliable from: https://sbim.org.br/images/files/manual-vigilancia-epidemiologica-eventos-vacinacao-4ed.pdf.
https://sbim.org.br/images/files/manual-...
).

Although professionals receive training to operationalize the IS-PNI/IS-AEFI, not all of them always feel able to fully operationalize the system(2929. Silva BS, Souza KC, Souza RG, Rodrigues SB, Oliveira VC, Guimarães EAA. Condições de estrutura e processo na implantação do Sistema de Informação de Imunização do Brasil. Rev Bras Enferm. 2020;73(4):e20180939. DOI: https://doi.org/10.1590/0034-7167-2018-0939.
https://doi.org/10.1590/0034-7167-2018-0...
). Issues related to the low training of professionals in surveillance and notification of AEFI can lead, therefore, to a large number of undefined cases, which compromises the real situation of AEFI-related IE, affecting care and cases progression(2828. Moro PL, Zheteyeva Y, Barash F, Lewis P, Cano M. Assessing the safety of hepatitis B vaccination during pregnancy in the Vaccine Adverse Event Reporting System (VAERS), 1990–2016. Vaccine. 2018;36(1):50-4. DOI: https://doi.org/10.1016/j.vaccine.2017.11.039.
https://doi.org/10.1016/j.vaccine.2017.1...
). In this regard, to ensure immunization quality, reliability, and safety, the improvement of surveillance actions and the use of notification forms are recommended(3030. Santos MCS, Netto VBP, MS Andrade. Prevalência e fatores associados à ocorrência de eventos adversos pós-vacinação em crianças. Acta Paul Enferm. 2016;29(6):626-32. DOI: https://doi.org/10.1590/1982-0194201600088.
https://doi.org/10.1590/1982-01942016000...
).

Due to the use of the IS-AEFI/IS-PNI database, the present study has limitations that are inherent to those based on secondary data. The possibility of over- or sub-notification of the occurrence of AEFI is highlighted. In addition, there is the possibility of incompleteness in the form of the recommended investigation sheet and errors in typing and filling of some fields. In this study, 99.57% of IE with no adverse events had the filling of “medical care” variable ignored, against 33.33% of IE with adverse events.

Furthermore, this work does not include AEFI notifications present in the Notification System in Health Surveillance, used for vaccination in private services.

CONCLUSION

The results of this study raise reflection on the reasons for the occurrence of IE in pregnant women and highlight the importance of continuing education to update the protocols and knowledge of the multidisciplinary team, especially the nurse, about immunization.

There is also an alert to the multidisciplinary team professionals, as a possibility to reduce missing data in the database, as well as better monitoring of professionals who fill out the notifications.

REFERENCES

Publication Dates

  • Publication in this collection
    01 Oct 2021
  • Date of issue
    2021

History

  • Received
    11 Dec 2020
  • Accepted
    29 June 2021
Universidade de São Paulo, Escola de Enfermagem Av. Dr. Enéas de Carvalho Aguiar, 419 , 05403-000 São Paulo - SP/ Brasil, Tel./Fax: (55 11) 3061-7553, - São Paulo - SP - Brazil
E-mail: reeusp@usp.br