Reliability of information available on popular websites about vaccination of pregnant women

1 Universidade Federal de Minas Gerais, Escola de Enfermagem, Belo Horizonte, MG, Brazil. 2 Universidade Federal de Minas Gerais, Escola de Enfermagem, Programa de Pós-Graduação em Enfermagem, Belo Horizonte, MG, Brazil. 3 Universidade Federal de Minas Gerais, Faculdade de Medicina, Programa de Pós-Graduação em Ciências da Saúde, Saúde da Criança e do Adolescente, Belo Horizonte, MG, Brazil. 4 Universidade Federal de Minas Gerais, Escola de Enfermagem, Departamento de Enfermagem Materno Infantil e Saúde Pública, Belo Horizonte, MG, Brazil. ABSTRACT Objective: To analyze the reliability of information available on popular websites about vaccination of pregnant women according to the recommendations of the Brazilian Ministry of Health. Method: Descriptive and comparative study. For data collection, a checklist composed of information on recommended, contraindicated, and indicated vaccines in special situations during pregnancy, according to the Ministry of Health, was elaborated. Results: None of the analyzed websites presented all the recommended information. Contraindications, most common adverse events, simultaneous administration of vaccines, information on the DT vaccine, and recommended vaccines in special situations were presented by a minority of websites. Conclusion: Information available on websites about the vaccination of pregnant women is not always based on the recommendations and misinformation may interfere with the acceptance of this practice. The importance of the professionals of the multidisciplinary team as information mediators, particularly the nurse, is emphasized, as is the need for regulating the production and dissemination of information on the internet.


INTRODUCTION
The decisions made by women during pregnancy may influence their health and their child's health (1)(2) . The changes in the immune system during pregnancy increase the risk of developing diseases, some of which have available immunization options (2)(3) . Vaccination during this period is therefore an efficient and safe mean of protecting woman and baby from certain diseases which are preventable through vaccination. Although some vaccines can be administered during pregnancy, pregnant women are still concerned and misinformed about the reliability of this method for preventing immunizable diseases (2)(3) .
According to recommendations by the Brazilian Ministry of Health (MH), vaccines against influenza, diphtheria, tetanus, pertussis, and hepatitis B are indicated during pregnancy. Some immunobiologicals, such as vaccines against yellow fever and rabies, are recommended in special situations. There are also vaccines that should not be administered during pregnancy, such as MMR, HPV, varicella, and dengue fever (4) . Studies report that women may face difficulties related to receiving the immunobiologicals which are recommended during pregnancy (5)(6) . According to the World Health Organization (WHO), "vaccinal hesitation refers to a delay in acceptance or refusal of vaccines in spite of vaccine availability in services [...] it is complex and context-specific, varying through time, place, and vaccines" (7) . Vaccinal hesitation in pregnant women is mainly related to insecurity regarding vaccine safety, misunderstanding, and absence of recommendation by health professionals (5)(6) .
The internet is currently one of the main means of access to information by the population, including pregnant women (8) . In Brazil, access to the network has been increasing remarkably and has been following a trend of growth, reaching 70% of Brazilians in 2018. The data revealed that 76% of Brazilian women are Internet users, 89% of them connect to the network every day, and 97% use the smartphone as their main access device. Regarding behavior when searching for information on the internet, 48% of Brazilian women are estimated to search for information related to health or health services in search websites on the internet (8) .
The search for information through the internet has been growing considerably among the pregnant population. A recent study performed with pregnant women in Turkey has pointed out that 48.5% of them resorted to the internet to search for diverse information related to pregnancy. In this research, the internet was described as the main source of information about this period (82.7%), followed by consultation of health professionals (68.4%) and friends (55.8%) (9) . The interpretation of information found on the internet is highly variable and dependent on the social and cultural perspectives of pregnant women (10)(11) . Also, quality of information is a problem on the internet, since it is not always reliable (12) . The use of the internet by pregnant women may bring benefits; however, inaccurate information made available on the network may negatively influence their decisions during prenatal, birth, and postpartum (12) .
Given the undefined quality of information on vaccines during pregnancy available on the internet and the important access of pregnant women to this information, in addition to the scarcity of studies on this theme, the objective of this study was to analyze the reliability of the information available on popular websites about vaccination of pregnant women, according to recommendations by the MH.

Design of stuDy
This is a descriptive and comparative study (13) with MH documents based on popular websites with the highest likelihood of being visited by pregnant women and which had information on vaccination of pregnant women. The design of study and the methodological steps were inspired by Monteiro et al. (13) .

LocaL
The websites were searched for on the search engine Google, due to it being widespread and easy to access (8) .

seLection criteria
The inclusion criteria of this study were websites targeted at laypersons and, for this reason, blogs, portals, media reports, private laboratory websites offering vaccination services and other websites related to vaccination and maternity were considered. The exclusion criteria were websites that did not approach vaccination during pregnancy, websites targeted at health professionals, by governmental institutions, and unavailable during data collection.
sampLe Sample selection was divided into two phases. The first took place in December 2019 and consisted of a simulation based on circumstantial search by a common pregnant woman, considering the most relevant websites as per the search engine. The following Portuguese terms have been employed separately: "vacinação em gestante" (vaccination in pregnant woman), "vacinação em mulheres grávidas" (vaccination in pregnant women), and "vacinação durante a gestação" (vaccination during pregnancy), and all websites in the first 10 pages for each term have been included. The second phase was performed from January to March 2020 and consisted of the assessment of the websites according to the central theme of their information, considering the inclusion and exclusion criteria.
The sampling and data collection phase were performed by two nursing academic researchers independently, supervised by a specialist nurse for two years on vaccination of pregnant women and researcher in epidemiology and health policies and practices of populations.
A total of 305 links have been found in circumstantial research, 14 duplicated websites have been removed, and 220 have been excluded after a thorough analysis of information due to not fitting the inclusion and exclusion criteria. After the selection, 71 websites were considered for checklist application ( Figure 1).

Data coLLection
The data collection was based on a checklist elaborated by the researchers containing important information on vaccination during pregnancy, which should be completely available and be easy to understand by laypersons. For structuration, the following recommendations of the MH were used: "Manual of Vaccination Norms and Procedures" (4) , "Technical Report for the Implementation of Adult Diphtheria, Tetanus, and Acellular Pertussis Vaccine Adsorbed -dTpa" (14) and "Manual of Epidemiological Surveillance of Postvaccination Adverse Events" (15) .
The structure of the data collection instrument was composed of relevant information about the vaccines recommended during pregnancy (influenza, hepatitis B, dT, and dTpa), assessed according to the following criteria: 1) Objective: influenza, to protect against infection caused by the influenza virus or against complications of this disease; hepatitis B, to protect against infection caused by the hepatitis B virus or against complications of this disease; dT, to protect against tetanus and diphtheria or against complications of these diseases; dTpa, to protect against diphtheria, tetanus, and pertussis or against complications of these diseases. 2) Number of doses: influenza, one dose; hepatitis B, 3 doses; dT, 3 doses; dTpa, one dose.
3) Ideal gestational age for administration: influenza, hepatitis B, and dT, in any gestational age; dTpa from the 20th or 27th week of pregnancy. 4) Contraindications of the recommended vaccines: pregnant women with allergy to some vaccine component. 5) Most prevalent postvaccination adverse events for the recommended vaccines: local reactions, such as pain, high temperature, swelling, hardening, reddening, and increased sensibility. 6) Simultaneous administration of the recommended vaccines: the recommended vaccines can be administered simultaneously. The following were also assessed: 7) Contraindicated vaccines: MMR, HPV, varicella, and dengue fever. 8) Vaccines indicated in special situations: yellow fever and rabies. The found information were classified as present, absent, divergent, or incomplete, and might be classified in some cases in more than one category.
Also, websites presenting references, publication and update dates, cohesion mistakes, and misconfigured content have been taken into account. Websites with contradictory information, such as administration always preferably before pregnancy, administration of vaccines only with medical recommendation and on the ideal gestational age for vaccine administration were also taken into account.

Data anaLysis anD treatment
After collection, the data were typed independently and compared using the program Epi Info (version 3.5.1) Figure 1 -Flowchart of the process of selection of websites for checklist application. and the identified divergences were discussed between the authors and decided by the nurse/researcher.
For data analysis, the package Statistical Software for Professionals (Stata), version 14.0 was used and the proportions were described according to the following classification: (I) correct information -if in accordance with the MH, (II) absent information -if not presented, (III) incorrect information -if not in accordance with the MH and (IV) incomplete information -if partially presented. In some cases, the information was classified as belonging to more than one category.

ethicaL aspects
Since the data were collected from websites available through a search engine with the use of public domain information, approval by an ethics committee was not required, according to resolution n. 510/2016 (16) .

RESULTS
Out of the 71 analyzed websites, none presented complete information in accordance with the recommendations of the Brazilian MH (4,(14)(15) . In relation to the influenza vaccine, most sites presented its objective correctly (71.83%). The number of recommended doses was omitted by 51 (71.83%) links, and only 20 (28.17%) presented the correct information. The ideal gestational age for administration of the immunobiological was correct and absent in the same proportion in 33 (46.48%) and incorrect in 5 (7.04%) websites (Table 1).
Concerning the information on the hepatitis B vaccine, 40 (56.34%) websites presented the information about the objective of the vaccine correctly and 31 (43.66%) omitted them. The information on the number of doses was correct in 40 (56.34%) websites; however, in 31 (43.66%), the information was absent. The gestational age was not presented in 38 (53.52%) websites and 30 (42.25%) presented it incorrectly ( Table 1).
The more commonly neglected information concerned the dT vaccine, for which, in all assessed items, the information was prevalently absent. The objective of this vaccine was not presented in 51 (71.83%) websites. The number of doses was absent in 48 (67.61%) websites. Also, 62 (87.32%) of them did not present the recommended gestational age for the administration of this immunobiological Table 1 -Analysis of the information collected from popular websites about vaccines recommended for pregnant women according to the manuals of the Ministry of Health -Brazil (4,(14)(15) .

Influenza
Hepatitis B dT dTpa
The contraindicated and recommended vaccines in special situations were also frequently neglected or incorrectly presented. The vaccines which are contraindicated during pregnancy were not mentioned by 35 (49.30%) websites and in 17 (23.95%) they were indicated incorrectly. Concerning the vaccines indicated in special situations, the information was absent in 32 (45.07%) websites and incorrect in 31 (43.66%) ( Table 2).
Some websites presented no date of information publication or update. Thirty-four websites presented date of publication: 16 disseminated the information in 2019, 8 in 2018, and 10 between 2008 and 2017. Also, 10 websites presented updates: 6 in 2019 and 1 in 2020, 2017, 2016, and 2013. A frequently presented information was in accordance with an outdated manual, pointing out that the ideal just gestational age for administration of the vaccine against hepatitis B is from the sixth month of pregnancy onwards, which was invalidated by a MH document published in 2014 (4) . Similarly, part of the websites indicated that pregnant women should receive more than one dose of dT in case she had been exposed to tetanus and had been vaccinated over five years ago. This recommendation became unnecessary and was changed with the inclusion of dTpa in the vaccine schedule of pregnant women (14) . Another update problem was that the dTpa vaccine was missing in some websites, as it was the last to be included in the vaccine schedule of pregnant women and started to be indicated from the publication of the document in 2014 (14) .
In some websites, gestational age was presented contradictorily: 2 (2.82%) websites informed in the beginning of the text that vaccination should be administered always during the first three months of pregnancy and 5 (7.04%) that vaccines should be applied only after the 3 rd month. On the other hand, further in the text, the gestational age was presented as recommended. In addition, 2 (2.82%) websites had textual cohesion errors that made text comprehension impossible and 1 (1.41%) was misconfigured, presenting superposed sentences and tables, hindering information legibility.
Other misinformation has been observed: 10 (14.08%) websites informed that vaccination should be preferably performed before pregnancy and 18 (25.35%) that vaccination should only be performed with medical recommendation. Also, 48 (67.61%) websites had no reference for the information about vaccination.

DISCUSSION
Contemplating the objective of analyzing the reliability of information available on popular websites about vaccination of pregnant women according to recommendations by the MH, this study has shown that popular websites used by pregnant women for searching for this information presented them unsatisfactorily, considering the important flaws which have been identified regarding quality of information. This situation is alarming and even more serious in the contemporary global context, due to the decline of vaccination coverage rates, the comeback of diseases which are preventable through vaccination, and the thriving of the anti-vaccine movement (17) .
Concerning the influenza vaccine, the results show that its objective was the information provided correctly more commonly and that the number of doses was absent in most of the websites. The MH suggests vaccination against influenza for protecting against the different strains of the virus influenza and the complications due to this infection (4) . Women should be vaccinated in each pregnancy at any moment, since its administration is safe for all gestational ages (18) , with attention to guarantee protection during the seasonal period being necessary (19)(20)(21) .
This study has also observed that the objective and the dose of the vaccine against hepatitis B were frequently provided correctly, whereas the ideal period for the administration of the vaccine was not presented or was approached incorrectly by almost all websites. According to the "Manual of Vaccination Norms and Procedures" (4) , this immunobiological may be administered to any age group and gestational age, depending on the record of the vaccine situation of pregnant women. This vaccine is efficient for immunization of the pregnant woman and her child, given that the acquired antibodies are also transmitted to the fetus (22)(23) .
The results have shown an expressive absence of information related to the dT vaccine. However, the objective was the information presented more often. The dTpa vaccine had the objective and gestational age presented correctly by most of the websites, whereas the number of doses was absent for most of them. The expressive absence of information about the dT vaccine may be related to the fact that this vaccine is recommended considering the vaccinal record of the patients. The vaccinal scheme of dT is composed of three doses and one of them may be provided with the dTpa vaccine. The dTpa vaccine is recommended for all pregnant women regardless of their vaccinal record due to protecting against pertussis.
A study has shown that the vaccination of pregnant women against influenza and dTpa is generally portrayed positively; however, some websites questioned the safety and efficacy of the vaccine against pertussis, prevented by the dTpa vaccine (4,24) . This information may have a negative impact on the vaccination of pregnant women, considering that the fear of possible damage and misconceptions about the need and efficacy of the vaccine are frequently mentioned as one of the main reasons for refusing vaccination (24) . Also, with no appropriate guidance, the information may have a maleficent effect for pregnant women, as it can cause confusion and trigger unjustified anxiety (12) .
This study has also shown that contraindications, postvaccination adverse events, and simultaneous administration were not correctly presented by most of the analyzed websites. Exaggerated and even fabricated versions of adverse events are very commonly one of the main sources for fostering the anti-vaccine movement (25) . According to the recommendations of the MH, the recommended vaccines are contraindicated in case of allergy to some component of the formula and, in these cases, the health professional must consider whether there is contraindication for subsequent doses or if the schema of administration should be changed or a different vaccine should be used in substitution (13) .
The vaccines which are recommended in special situations were not mentioned as per the recommendations by any of the websites; nonetheless, for most websites, the information was absent or divergent. Most sites did not present any information on the vaccines which are contraindicated during pregnancy. According to the MH, the administration of the recommended vaccines in special situations must be indicated in cases of exposure or high risk of developing these diseases (4) . Regarding the contraindicated ones, the adverse effects for the fetus are not established, but there are studies showing that pathogens of live attenuated vaccines can replicate in the individual, causing virulence (26) . There is thus the theoretical possibility of vaccination during pregnancy leading to severe adverse consequences for mother and fetus, outweighing the beneficial effect of the vaccine (4) .
Pregnant women use the internet not only to search for information about pregnancy, but also for emotional support, professional support, connection with other women and couples, and entertainment (12) . With the information they find, pregnant women feel more confident to make decisions, reducing their anxiety, uncertainties, and feeling of isolation (12) . There is preoccupation about information related to pregnancy, given that it is not always reliable (12) and the regulation of health information is scarce or nonexistent (27) .
A recent study has identified in social networks the predominance of positive information on vaccines and interest by the population on themes related to scientific research. However, the topics were more explored by variety and journalistic websites, and there was little participation of academic and scientific institutions and none by governmental bodies. Also, 8 of the 100 links presented information against vaccination, 7 of which were fake news, showing that their dissemination is an important tool of the anti-vaccine discourse (28) .
There is preoccupation with the pregnant women's capacity of evaluating information accuracy, given that pregnant women usually consider information they have found as useful and reliable, not discussing their findings with health professionals (12) . Since the role of the internet in vaccination practices is still not completely clear, information related to this theme and to the influence of social media should be better monitored (7) .
Although women frequently search for information on the internet, health professionals continue to be a valued source of information for them (29) . The multidisciplinary team professionals, particularly nurses, are essential for the promotion of vaccination during pregnancy. The professional recommendations are an important encouragement for pregnant women and an important predictor for acceptance of vaccination. This study's results emphasize the importance of health education, with the incorporation of practices that encourage women to use reliable sources of information, discuss findings with reference professionals, and being exposed to information on the benefits of vaccination for the mother-infant dyad, in addition to stimulating women in fertile age to maintain their vaccination card up to date.
Limitations of this study include not having employed a design systematic assessment, measurement of website coverage and browsing, the use of a single search engine, and not having included other vehicles of access to information, such as social networks. In addition, the adopted instrument was not validated, since the studied object focused information by the Brazilian ministry to subsidize comparison of the selected links.

CONCLUSION
This study has shown that the quality of information available on popular websites about vaccination of pregnant women is questionable, since in all of the analyzed aspects, important gaps have been observed and none of the websites presented all the information recommended by the MH, which may lead to interferences in the acceptance of this www.scielo.br/reeusp Rev Esc Enferm USP · 2021;55:e20200517 practice. Contraindications, most common postvaccination adverse events, simultaneous administration of vaccines, information about dT vaccines and those recommended in special situations have been neglected by most of the websites.
Multidisciplinary work in this context, particularly the nurse's, is extremely important to intermediate information, and the need for regulating the production and dissemination of information about health on the internet is also noteworthy.