Evaluation of the Brazilian surveillance system for adverse events following vaccination

ORIGINAL ARTICLES ARTIGOS ORIGINAIS

Evaluation of the Brazilian surveillance system for adverse events following vaccination

Avaliação do sistema brasileiro de vigilância de eventos adversos pós-vacinação

Sandra Aparecida Moreira Gomes Monteiro^I; Olga Akiko Takano^I; Eliseu Alves Waldman^II

^IInstituto de Saúde Coletiva da Universidade Federal de Mato Grosso - Cuiabá (MT), Brasil

^IIFaculdade de Saúde Pública da Universidade de São Paulo - São Paulo (SP), BrasilTrabalho realizado no Instituto de Saúde Coletiva da Universidade Federal de Mato Grosso - Cuiabá (MT), Brasil

Correspondência

ABSTRACT

OBJECTIVES: To describe and evaluate the Brazilian system of passive surveillance of adverse events following immunization (PSAEFI).

METHODS: The description and evaluation of PSAEFI were undertaken using the reported cases of adverse events following immunization with DTwP-Hib vaccine (AEFI-T), during the period from 2002 to 2005, using the Centers for Disease Control methodology.

RESULTS: The PSAEFI system, which provides national coverage, is designed to standardize practices in cases of adverse events following immunization (AEFI) and to identify highly reactogenic lots of vaccine. The PSAEFI system proved its usefulness, simplicity and flexibility; despite low sensitivity, overestimate the proportion of sever events, but it consistently described AEFI-T, identifying fever, convulsions and hypotonic-hyporesponsive episodes as the most common events. It showed that 49.7% of AEFI-T occur after the first dose, and that 72.8% occur within the first six hours after vaccination. It facilitates public health decisions and epidemiological investigations. It is timely, 46.1% of all AEFI-T being reported within 10 days after vaccination and its completeness ranges from 70 to 90%, depending on the item evaluated.

CONCLUSIONS: The PSAEFI system proved useful for monitoring DTwP-Hib vaccine safety. We recommended the incorporation of new methodologies, such the use of sentinel cities/hospitals and computerized immunization registries in order to increase its sensitivity.

Keywords: immunization program; vaccination; product surveillance, postmarketing; epidemiologic surveillance; clinical trial, phase IV.

RESUMO

OBJETIVOS: Descrever e avaliar o Sistema brasileiro de vigilância passiva de eventos adversos pós-vacinação (SPVEAPV).

MÉTODOS: A descrição e avaliação do SPVEAPV fundamentaram-se nas notificações de eventos adversos pós-vacina Tetravalente ou DTwP/Hib - vacina adsorvida difteria, tétano, pertussis e Haemophilus influenzae b (conjugada) - (EAPV-T), durante o período de 2002 a 2005. Empregou-se a metodologia proposta pelo Centers for Disease Control and Prevention.

RESULTADOS: O SPVEAPV apresenta abrangência nacional, tem por objetivos identificar e padronizar condutas frente a casos de eventos adversos pós-vacinação (EAPV) e identificar lotes reatogênicos. A vigilância é útil, simples e flexível, sua sensibilidade é baixa, superestima os eventos mais graves, mas descreve de forma consistente os EAPV-T, identificando a convulsão, a febre e o episódio hipotônico-hiporresponsivo como os mais frequentes, apontando a maior proporção de EAPV na primeira dose (49,7%) e nas primeiras seis horas após a vacinação (72,8%); é útil ao subsidiar decisões e investigações complementares; 46,1% das notificações são feitas até 10 dias após a vacinação; sua completude varia de 70% a 90%, conforme o item.

CONCLUSÕES: O SPVEAPV mostra-se útil no monitoramento da segurança da vacina DTwP/Hib, sendo, porém, recomendável a incorporação de novas metodologias como a de municípios e/ou hospitais sentinelas e a de sistemas informatizados de registros de imunização com a finalidade de elevar sua sensibilidade.

Palavras-chave: programas de imunização; vacinação; vigilância de produtos comercializados; vigilância epidemiológica; ensaio clínico fase IV.

Introduction

The Brazilian National Immunization Program (NIP), created in 1973, was a landmark in the history of public health in Brazil due to its high degree of organization and success in controlling diseases that, at that time, were responsible for high rates of infant morbidity and mortality^1,2.

Like other countries^3,4, in view of concerns with vaccine safety and attempting to maintain high rates of adherence to the routine vaccination, Brazil implemented in 1998 a passive surveillance for adverse events following immunization. The Brazilian system of passive surveillance of adverse events following immunization (PSAEFI) is a nationwide surveillance system designed to monitoring vaccine safety and timely investigating suspected adverse events to support appropriate actions⁵.

Besides the inherent limitations of passive surveillance for adverse events following immunization⁶, the large territorial area of Brazil and its regional differences are additional challenges to the operationalization of the PSAEFI if compared to those avaible in a developed country.

The periodic evaluation of a surveillance system is a procedure that allows the degree of usefulness of the information provided by the surveillance to be determined, as well as allowing the event of interest to be characterized in terms of its magnitude, the risk of its occurrence to be estimated and the most vulnerable groups to be identified⁷.

The introduction of the combined diphtheria-tetanus-whole-cell pertussis and Haemophilus influenzae type b (DTwP/Hib) vaccine⁸ into the Brazilian routine immunization schedule in 2002 offers an opportunity to evaluate the performance of the PSAEFI. The DTwP/Hib vaccine was chosen as it is supplied by a single manufacturer⁹ and has been associated with most AEFI reports in Brazil¹⁰.

The objective of this study was to describe the BSAEFI and evaluate its performance indicators during 2002 to 2005.

Materials and Methods

This was a descriptive study in which the study area comprised the entire Brazilian territory including 26 states and one federal district, a total area of about 8.5 million km² and 190 million inhabitants¹¹. The study population included infants less than one year of age immunized with at least one dose of DTwP/Hib vaccine during 2002 to 2005. During the period of interest, the estimated population of infants less than one year of age in Brazil ranged from 3.3 million (in 2002) to 3.5 million (in 2005)¹¹.

The DTwP/Hib (or tetravalent) vaccine used in Brazil has been produced by Oswaldo Cruz Foundation (FIOCRUZ) Immunobiological Technology Institute (Bio-Manguinhos) in partnership with the Butantan Institute, which produces the DTwP fraction of the vaccine. The vaccine specifications are described elsewhere⁹.

Information was obtained from the PSAEFI on reported cases of adverse events following DTwP/Hib vaccine (AEFI-T); the Brazilian Ministry of Health National Health System Database (DATASUS) on vaccine doses and coverage rate; and the Brazilian Institute of Geography and Statistics (IBGE) on population data. The AEFI-T case database is stored as part of the general PSAEFI database, denominated AEFI Information System (AEFI-IS).

The PSAEFI was evaluated according method proposed by the Centers for Disease Control and Prevention (CDC)⁷. The magnitude and relevance of the event were described as well as the following surveillance components: objectives; case definition; sources of information; type of surveillance; analysis; and information dissemination. The same method was applied to assess qualitative and quantitative performance indicators.

The following concepts were used:

Types of surveillance:

- Passive surveillance: information is obtained by spontaneous reporting;

- Active surveillance: information is obtained on a regular basis through direct contact between the surveillance team and information sources, or through electronic immunization records as part of electronic medical records⁶.

Qualitative indicators:

- Usefulness: analyzes the degree to which the surveillance achieves its objectives, estimating the magnitude of AEFIs with the DTwP/Hib vaccine and whether it contributes to increasing the body of knowledge regarding the issue, identifying changing trends and stimulating research;

- Simplicity: evaluates the structure and its ease of operation;

- Flexibility: analyzes the ability of a surveillance to readily adapt to new information needs in response to changes in the nature or importance of the event of interest;

- Acceptability: evaluates compliance with the surveillance on the part of health professionals and health care facilities, one of the parameters being the completeness of the information included on the reporting form.

Quantitative indicators:

- Timeliness: evaluates the speed at which each step established by the surveillance system is achieved;

- Sensitivity: measures the proportion of true cases identified by the surveillance.

The consistency of the PSAEFI-T data was assessed based on case definition (presented and discussed in Results and Discussion). Only PSAEFI confirmed cases occurring among infants less than one year of age were included; cases associated with other vaccines, and those that were still under investigation were excluded. Since hypotonic-hyporesponsive episodes (HHEs) can be confused with convulsions, all AEFI-T involving HHEs accompanied by convulsion were registered as cases of convulsion alone¹².

A severe AEFI-T case was defined as any adverse event following DTwP/Hib vaccine that has resulted in death, hospitalization longer than 12 hours as well as cases of HHE, convulsions, encephalopathy, purpura, and hypersensitivity reaction within 2 hours after vaccination^13,14.

For calculation of rates of reported cases the number of doses administered was used as the denominator. When there was more than one AEFI-T reported for the same child and dose, these events were considered as a single case with two or more events.

This study was approved by the Research Ethics Committee at Júlio Müller University Hospital at Universidade Federal de Mato Grosso (protocol nº. 276/CEP-HUJM/06).

Results

Magnitude of the event and system description

Between 2002 and 2005, approximately 34 million doses of the DTwP/Hib vaccine were administered nationwide in Brazil. Administrative data show that, during the study period, the mean annual coverage among infants less than one year of age for the third dose of the DTwP/Hib vaccine was 85.0% nationwide (range, 66.0-100.0% among states), varying between 81.0% and 89.0% in the macroregions evaluated (north, south, northeast, central-west and southwest)¹⁵.

There were 14,241 reported DTwP/Hib vaccine AEFI, corresponding to 11,558 infants, based on the case definition and inclusion/exclusion criteria adopted. The mean annual number of reported cases during the study period was 2,890, with a trend toward growth, increasing from 2,126 in 2002 to 3,383 in 2005.

The BSAEFI objectives included: i) to develop standards for identification of AEFI cases and management; ii) to analyze the national AEFI database; iii) to identify highly reactogenic batches and make decisions regarding management; iv) to ensure the population and health professionals that reliable immunobiologicals are used in the NIP⁵.

Created by the NIP in 1998, the PSAEFI is a nationwide passive surveillance that receives reports from both public and private primary health care and hospital settings, especially from public ones. First restricted to the NIP, since 2008 the PSAEFI joint activities and promoting cooperation whit the Brazilian Regulatory Health Agency (ANVISA) and the Oswaldo Cruz Foundation National Institute of Health Quality Control (INCQS)

According to the PSAEFI a confirmed AEFI-T case is defined as any infant less than one year of age who following any dose of DTwP/Hib vaccine had one or more adverse events reported to the AEFI passive surveillance. Mild AEFI-T cases and those occurring more than 72 hours following immunization are excluded, with the exception of encephalopathy, which is considered to be an AEFI-T if identified within the first week after vaccination^5,16.

The reports are made at the local level using a specific PSAEFI form. These forms are passed on to the state level on a monthly basis or immediately in the case of severe or uncommon events or outbreaks. Data are analyzed by trained technical staff at the state level. All confirmed cases are registered in the AEFI Information System (AEFI-IS) and passed on to the NIP on a monthly basis (Figure 1). The AEFI Information System was especially designed to store information on reported AEFI cases collected on standard reporting forms. It is used for recording and passing on reports from states to the NIP and for data analysis.

The NIP has, on a regular basis, updated the guidelines containing the technical norms employed for the training of personnel and the standardization of procedures^5,16. One of those guidelines is designed exclusively for training local health care teams that work at vaccination sites¹⁷.

Evaluation of the system's performance indicators

Data usefulness

The BSAEFI proved its usefulness. It was able to estimate the magnitude of the most common AEFI-T including seizure, fever and HHE (Table 1). It described their main features, showing a predominance of events following the first dose (49.7%) compared with the second (35.1%) and third doses (15.2%). It was found that 13.6% of these events occurred within the first hours following vaccination, 59.2% within one to six hours and only 2.3% occurred after 72 hours (Figure 2).

The weekly analysis of data showed peaks of reports, especially in macroregions of the country, where the surveillance proved to have higher sensitivity (Table 2). However, when vaccine batches with a higher number of reports associated were individually analyzed, no association was found with these peaks as reports were distributed over time. The individual analysis showed that batches with the highest number of reports associated had a proportion of serious adverse events similar to the average occurrence in the remaining batches (HHE: 25% to 55.9%; fever: 11.8 to 45.7%; and convulsions: 0% to 20.6%). Another study¹³ has pointed out the difficulty to identify highly reactogenic batches making it necessary to devise new monitoring strategies.

The PSAEFI databases provide information that can be used in research studies to further explore these adverse events^18,19 and can provide input for the development and regular update of standards and regulations for safe vaccine use in Brazil.

Simplicity

The PSAEFI system is simple, the information flow being of low complexity, not involving laboratory tests or other specialized services (Figure 1)⁷. Another contributor to this simplicity is the fact that the public network of primary health care facilities not only functions as the principal source of reporting but is also responsible for the overwhelming majority of DTwP/Hib vaccinations carried out nationwide, as well as for most of the medical appointments related to mild or medium-severity AEFIs with the DTwP-Hib vaccine^18,19. However, there are aspects that confer a certain degree of complexity upon the system. The data analysis and case confirmation require the involvement of well-trained teams. In addition, the perfect characterization of an AEFI is elusive, whether due to the absence of pathognomonic signs or to the unavailability of laboratory tests that would allow confirmation of the diagnosis^3,4,6. Nevertheless, the identification of a sign that could suggest an as-yet-unidentified adverse event requires the involvement of teams trained to carry out more complex investigations⁶.

Flexibility

The BSAEFI proved to be flexible, since the introduction of the DTwP/Hib vaccine, as a replacement for the DTwP vaccine, did not alter the costs, routines, data sources or performance of the system¹⁹.

Acceptability

Of a total of 5,564 municipalities, 1,999 (35.9%) reported an AEFI at least once during the study period (Table 2), and of them only 230 (4.1%) did it at least once a year. However, only 11 out of 5,564 municipalities (0.2%) have a population of more than 20,000 infants less than one year of age, which reduces the likelihood of identifying rare events.

Since it is mandatory the completion of the fields related to patient identification (name, age, gender), and date of vaccine administration and reporting in order to register the event in the AEFI database, this information was complete in 100% of reports.

Information related to the evolution of the AEFI-T, the strategy adopted in relation to maintaining or suspending the vaccine schedule, the type of hospitalization, the name of the laboratory that produced the vaccine and the vaccine lot number was included, respectively, in 79.5%, 75.4%, 69.6%, 92.3% and 84.7% of the reports.

Timeliness

Based on nationwide averages, in 25.3% of cases (range: 21.8% to 32.5%) the time between vaccine administration and reporting was one day and in 46.1% (range: 41.5% to 53.3%) was up to 10 days. Nationwide, an AEFI was registered in the AEFI database on the same date, 15 and 30 days after the reporting in 17%, 20% and 27% of cases, respectively, but in no less than 60 days in 57% of cases.

Sensitivity

The nationwide average AEFI-T reporting rate was 41.9 per 100,000 doses administered (ranging from 6 to 141 in the different geographical areas studied). The nationwide average rates for HHE and convulsions were 15 per 100,000 doses (ranging from 40 to 70), and 6 per 100,000 doses (ranging from 1 to 17). A comparison of these average rates found in the present study with other well-known studies^20,21 showed that they are well below the expected, suggesting that the BSAEF has low sensitivity. This finding is corroborated by a recent study that estimated an average sensitivity of the BSAEFI-T for HHE and convulsions of 22.3% and 31.6% respectively; however, this same study reported widely varying rates, with sensitivity greater than 90% in some areas¹⁴.

Discussion

The study showed that the PSAEFI has an adequate performance despite its low sensitivity. The surveillance proved useful in identifying and consistently describing AEFI-T most often associated with vaccines containing the pertussis component^20,21,22.

The surveillance's good performance can be partly explained by the fact that Brazil has a broad network of primary health care facilities and a previously consolidated immunization program, with trained, experienced teams and up-to-date guidelines for training courses that are also offered to health professionals working in local health care units^1,5,16.

The study results showed that the PSAEFI has an appreciable degree of timeliness. Yet in approximately 50% of the AEFI-T, the time for register in the AEFI database was no less than 60 days. Despite this limitation the PSAEFI has successfully identified highly reactogenic vaccine batches^23,24. In addition, it has good acceptability⁶, though it may be overestimated given the requirements for to register the event in the AEFI electronic database.

The average sensitivity of the BSAEFI was low. Sensitivity rates varied among states and were higher in those with better socioeconomic indicators and quality health services. Higher sensitivity rates were not associated with lower vaccination coverage¹⁴, i.e., the fact that health professionals are more aware of AEFI occurrence has not negatively affect adherence to vaccination.

As seen in other countries²², increasing AEFI reporting rates over the study period may be a result of increased sensitivity of the PSAEFI due to system improvements and due to the growing perception of the risk of AEFIs in certain social strata of the society. A similar trend was also reported by Freitas¹³ in the state of São Paulo, southeastern Brazil, over a longer study period while analyzing adverse events following DPT vaccination before the introduction of DTwP/Hib vaccine.

AEFI case definition may also affect sensitivity of the BSAEFI. The case definition used in Brazil focuses mainly on more serious events with public health relevance such as HHE and convulsions without regard for mild events such as fever. It has the advantage, especially in developing countries, of making surveillance less complex and costly as it examines smaller sets of data¹³. However, it may overestimate the relative importance of more serious events^3,4 evidenced by fewer reports of fever than HHE while possibly underestimating late events, except for encephalopathy⁵.

The fact that the PSAEFI system is, in various aspects, a complex system of surveillance can produce favorable results, since it is a factor that induces the development of specialized human resources and the improvement of continuing education programs. In addition, this complexity can create conditions favorable to the strengthening of ties between the PSAEFI system and research groups, facilitating rapid responses in situations that elicit concern and anxiety on the part of the population in relation to vaccine safety⁹.

Many of the limitations identified in the Brazilian experience with the PSAEFI system are inherent to the use of passive surveillance, independent of the characteristics of the health care services in the country in which it is developed^3,4,7.

Perhaps the most important of such limitations is the low specificity, since the system frequently identified AEFIs temporally associated with vaccination but without a causal relationship having been established^7,25. Another limitation is its low sensitivity, as previously discussed^7,3.

The surveillance's limitations are also related to exposure to multiple vaccinations especially among infants less than one year of age; high number of potential AEFI⁴ as well as difficulty of identifying re-exposure to the same vaccine among individuals with previous reports of AEFI³. The choice for reporting more severe cases not only reduces the sensitivity of surveillance but also its representativeness²⁶.

The favorable performance that the NIP has achieved in Brazil is the result of a set of public policies that strengthen the three pillars that on which the program rests: broad coverage, equity in access and vaccine safety²⁷. These public policies have strengthened national vaccine manufacturers as well as quality control of vaccines, ensuring regular supply of biological products²⁷. The NIP has promoted scientific and technological development, including training of multidisciplinary research teams and continuing education^9,28,29,30. It has supported the conduction of periodic population-based surveys to monitor vaccine coverage and equitable access to vaccines³¹.

The study showed the usefulness of passive surveillance systems for AEFI despite their limitations and challenges faced by developing countries like Brazil with large territorial areas and wide regional differences. These data support passive surveillance for AEFI as a strategic component for high levels of adherence to vaccination as it is a key tool for ensuring vaccine safety and reliability of immunization programs³². They also indicate a need for new strategies to more effectively identify highly reactogenic vaccine batches as well as new methodologies such as sentinel hospitals/cities and electronic immunization records for increasing the system's sensitivity³³.

References

• ¹
  Brasil. Programa Nacional de Imunização - 30 anos. Secretaria de Vigilância a Saúde. Ministério da Saúde; 2003.
• 2. Waldman EA, Silva LJ, Monteiro CA. Trajetória das doenças infecciosas: da eliminação da poliomielite à reintrodução da cólera. In: Monteiro CA. Velhos e novos males da saúde no Brasil: a evolução do país e de suas doenças. 2a. ed. São Paulo: Hucitec, Nupens/USP; 2000. p. 195-244.
• 3. Singleton JA, Lloyd JC, Mootrey GT, Salive ME, Chen RT, VAERS Working Group. An overview of the vaccine adverse event reporting system (VAERS) as a surveillance system. Vaccine. 1999;17(22):2908-17.
• 4. Lawrence G, Menzies R, Burgess M, McIntyre P, Word N, Boyd I, et al. Surveillance of adverse events following immunization: Australia, 2000-2002. Commun Dis Intell. 2003;27(3):307-23.
• ⁵
  Brasil. Ministério da Saúde. Fundação Nacional de Saúde. Manual de vigilância epidemiológica dos eventos adversos pós-vacinação. Brasília; 1998.
• 6. Centers for Disease Control and Prevention (CDC). Iskande JK, Miller ER, Pless RP, Chen RT. Vaccine safety post-marketing surveillance: the vaccine adverse event reporting system. 2006.
• 7. Centers for Disease Control and Prevention - Updated guidelines for evaluating public health surveillance systems. Recommendations from the Guidelines Working Group - United States. MMWR. 2001;50 (Nº RR-13).
• 8. Araújo O, Forleo-Neto E, Vespa GNR, Puccini RF, Weckx LW, Carvalho ES, et al. Associated or combined vaccination of Brazilian infants with a conjugate Haemophilus infuenzae type b (Hib) vaccine, a diphtheria-tetanus-whole-cell pertussis vaccine and IPV or OPV elicits protective levels of antibodies against Hib. Vaccine. 2001;19(2-3):367-75.
• 9. Martins RM, Camacho LAB, Lemos MCF, Noronha TG, Carvalho MHC, Greffe N, et al. Incidência de episódios hipotônicos-hiporresponsivos associados à vacina combinada DTP/Hib usada no Programa Nacional de Imunizações. J Pediatr (Rio J). 2007;83(6):523-8.
• 10. Martins RM, Maia MLS. Eventos adversos pós-vacinais e resposta social. Hist Cienc Saúde - Manguinhos. 2003;10(2):807-25.
• ¹¹
  Brasil. Fundação IBGE - Instituto Brasileiro de Geografia e Estatística. Informações de saúde. Estatísticas vitais. Demográficas e sócio-econômicas. [internet]. [citado 02 jul 2006] Disponível em:< http://www.datasus.gov.br>
• 12. DuVernoy TS, Braun MM, VAERS Working Group. Hypotonic-hyporesponsive episodes Reported to Vaccine Adverse Event Reporting System (VAERS), 1996-1998. Pediatrics. 2000;106(4):52-61.
• 13. Freitas FRM. Vigilância de eventos adversos associados à vacina DPT e preditores de gravidade. Estado de São Paulo. 1984-2001. (Dissertação de Mestrado). São Paulo: Faculdade de Saúde Pública da Universidade de São Paulo; 2005.
• 14. Monteiro SAMG, Takano OA, Waldman EA. Surveillance for adverse events after DTwP/Hib vaccination in Brazil: Sensitivity and factors associated with reporting. Vaccine. 2010;28(18):3127-33.
• ¹⁵
  Ministério da Saúde. Departamento de Informação e Informática do Sistema Único de Saúde (DATASUS) [internet]. [citado 8 maio 2009] Disponível em:<http://www.datasus.gov.br>
  » link
• ¹⁶
  Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Manual de eventos adversos pós-vacinação. Brasília; 2008.
• ¹⁷
  Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Vigilância dos eventos adversos pós-vacinação: cartilha para trabalhadores de sala de vacinação. Brasília; 2003.
• 18. Monteiro SAMG. Avaliação dos eventos adversos pós-vacina Tetravalente. Brasil, 2002-2005. (Dissertação de Mestrado). Cuiabá (MT), Brasil: Instituto de Saúde Coletiva da Universidade Federal de Mato Grosso; 2007.
• 19. Freitas FRM, Sato HK, Aranda CMS, Arantes BAF, Pacheco MA, Waldman EA. Eventos Adversos pós-vacina contra a difteria, coqueluche e tétano e fatores associados à sua gravidade. Rev Saúde Pública. 2007;41(6):1032-41.
• 20. Baraff LJ, Shields WD, Beckwith L, Strome G, Marcy SM, Cherry JD, et al. Infants and children with convulsions and hypotonic-hyporesponsive episodes following diphtheria-tetanus-pertussis immunization: follow-up evaluation. Pediatrics. 1988;81(6):789-94.
• 21. Cody CL, Barraff LJ, Cherry JD, Marcy SM, Manclark CR. Nature and rates of adverse reactions associated with DPT and DT immunizations in infants and children. Pediatrics. 1981;68(5):650-60.
• 22. Chen RT. Evaluation of vaccine safety after the events of 11 September. 2001: role of cohort and case-control studies. Vaccine. 2004;22(15-16):2047-53.
• 23. Dourado I, Cunha S, Teixeira MG, Farrington CP, Melo A, Lucena R, et al. Outbreak of aseptic meningitis associated with mass vaccination with a urabe-containing measles-mumps-rubella vaccine: implications for Immunization programs. Am J Epidemiol. 2000;151(5):524-30.
• 24. Silva LJ. Vacinação, segurança de imunobiológicos e direitos do cidadão [editorial]. Rev Saude Publica. 1996;30(4):297-8.
• 25. Begg N, Miller E. Role of epidemiology in vaccine policy. Vaccine. 1990;8(3):180-9.
• 26. Heininger U. The success of immunization - shoveling its own grave? Vaccine. 2004;22(15-16):2071-2.
• 27. Waldman EA. Elevadas coberturas, equidade e segurança. Desafios do Programa Nacional de imunizações. Rev Bras Epidemiol. 2008;11(Suppl 1):129-32.
• 28. Martins RM, Bensabath G, Arraes LC, Oliveira MLA, Miguel JC, Barbosa GG, et al. Multicenter study on the immunogenicity and safety of two recombinant vaccines against hepatitis B. Mem Inst Oswaldo Cruz. 2004;99(8):865-71.
• 29. Camacho LAB, Aguiar SG, Freire MS, Leal MLF, Nascimento JP, Iguchi T, et al. Reatogenicidade de vacinas contra febre amarela em estudo randomizado, controlado com placebo. Rev Saúde Pública. 2005;39(3):413-20.
• 30. Ioshimoto LM, Rissato ML, Bonilha VSJ, Miyaki C, Raw I Granovsky N. Safety and immuunogenicity of hepatitis B vaccine Butang in adults. Rev Inst Med Trop São Paulo. 1999;41(3):191-3.
• 31. Moraes JC, Barata RB, Ribeiro MCSA, Castro PC. Cobertura vacinal no primeiro ano de vida em quatro cidades do estado de São Paulo, Brasil. Rev Panam de Salud Publ. 2000;8(5):332-41.
• 32. Duclos P, Delo A, Aguado T, Bilous J, Birminghan M, Kieny MP, et al. Immunization safety priority project at the World Health Organization. Seminar Pediat Infect Dis. 2003;14(3):233- 9.
• 33. Waldman EA, Luhm KR, Monteiro SAMG, Freitas FRM. Vigilância de eventos adversos pós-vacinação e segurança de programas de imunização. Rev Saúde Pública. 2011;45(1):173-84.

¹

.

Publication Dates

• Publication in this collection
  07 Nov 2011
• Date of issue
  Sept 2011

History

• Reviewed
  20 Jan 2011
• Received
  15 Mar 2010
• Accepted
  28 Feb 2011