YOUTUBE AS A SOURCE OF INFORMATION ABOUT UNPROVEN DRUGS FOR COVID-19: the role of the mainstream media and recommendation algorithms in promoting misinformation

YOUTUBE COMO FONTE DE INFORMAÇÃO SOBRE MEDICAMENTOS SEM EFICÁCIA COMPROVADA PARA COVID-19: o papel da imprensa tradicional e dos algoritmos de recomendação na promoção da desinformação

YOUTUBE COMO FUENTE DE INFORMACIÓN SOBRE MEDICAMENTOS NO PROBADOS PARA EL COVID-19: el papel de los principales medios de comunicación y los algoritmos de recomendación en la promoción de la desinformación

ABSTRACT

In this study, we address how YouTube videos promote misinformation about hydroxychloroquine in Brazil. We follow two research questions. RQ1: How is pro-hydroxychloroquine content propagated on YouTube? RQ2: How does YouTube’s recommendation system suggest videos about hydroxychloroquine on the platform? We use mixed methods (content analysis and social network analysis) to analyze 751 YouTube videos. We found that most pro-HCQ videos in our dataset are posted by mainstream media channels (RQ1) and that YouTube was more likely to recommend proHCQ videos than anti-HCQ videos (RQ2). Consequently, the Brazilian mainstream media and YouTube’s algorithms fueled the spread of pro-HCQ content.

Key words
Misinformation; Covid-19; YouTube; Echo Chamber; Mainstream Media

RESUMO

Neste estudo, abordamos como os vídeos no YouTube promovem desinformação sobre a hidroxicloroquina no Brasil. Seguimos duas questões de pesquisa. RQ1: Como o conteúdo pró-hidroxicloroquina é propagado no YouTube? RQ2: Como o sistema de recomendação do YouTube sugere vídeos sobre hidroxicloroquina na plataforma? Usamos métodos mistos (análise de conteúdo e análise de redes sociais) para analisar 751 vídeos do YouTube. Descobrimos que a maioria dos vídeos pró-HCQ em nosso conjunto de dados foi postado pelos principais canais de mídia (RQ1) e que o YouTube era mais propenso a recomendar vídeos pró-HCQ do que vídeos anti-HCQ (RQ2). Consequentemente, a imprensa tradicional brasileira e os algoritmos do YouTube fomentaram a disseminação de conteúdo pró-HCQ.

Palavras-chave
Desinformação; Covid-19; YouTube; Câmaras de Eco; Imprensa Tradicional

RESUMEN

En este estudio, abordamos cómo videos en YouTube promueven desinformación sobre la hidroxicloroquina en Brasil. Seguimos dos preguntas de investigación. RQ1: ¿Cómo se propaga el contenido pro-hidroxicloroquina en YouTube? RQ2: ¿Cómo el sistema de recomendaciones de YouTube sugiere videos sobre hidroxicloroquina en la plataforma? Utilizamos métodos mixtos (análisis de contenido y análisis de redes sociales) para analizar 751 videos de YouTube. Descubrimos que la mayoría de los videos pro-HCQ en nuestro conjunto de datos son publicados por los principales canales de medios de comunicación (RQ1) y que era más probable que YouTube recomendara videos pro-HCQ que videos anti-HCQ (RQ2). En consecuencia, los principales medios de comunicación brasileños y los algoritmos de YouTube impulsaron la difusión de contenido pro-HCQ.

Palabras clave
Desinformación; Covid-19; YouTube; Cámara de Eco; Principales Medios de Comunicación

1 Introduction

YouTube is a very popular platform worldwide, however, it is often used for promoting misleading information about health (Madathil et al., 2015). In Brazil, YouTube is the most popular social media platform for news consumption as 43% of the online population regularly access it focusing on news consumption (Newman et al., 2022). In the context of the covid-19 pandemic, Brazilian YouTube channels have been spreading misinformation claiming that the virus was not a threat, promoting conspiracy theories, and criticizing social distancing and other preventative measures (Machado et al., 2020). Due to the popularity of YouTube, videos containing misleading information about covid-19 represented a problem for reducing the spread of the virus, as they fuel the so-called “infodemic” (Tangcharoensathien et al., 2020).

In this study, we look at YouTube videos about hydroxychloroquine and chloroquine in Brazil. Although these drugs are not recommended as a treatment or prophylaxis for covid-19, Brazilian President Jair Bolsonaro and many of his supporters have been promoting hydroxychloroquine and chloroquine as a “cure” and prophylaxis for covid-19 (Araujo & Oliveira, 2020; Soares et al., 2020; Casarões & Magalhães, 2021). It is important to highlight that Brazil faces a complex scenario of political-ideological polarization derived from a neoconservative wave that has been increasing since 2014 (Alves, 2022; Soares et al., 2019). This far-right movement became more acute after Bolsonaro’s victory in the 2018 presidential elections (Watmough, 2021).

Therefore, we aim to analyze videos about hydroxychloroquine on YouTube, in particular, videos that reproduce content promoting the drugs; and explore how YouTube’s recommendation algorithm might contribute to the spread of misinformation and the emergence of echo chambers. We use content analysis and social network analysis to explore a dataset of 751 videos collected via the YouTube API v3 (Rieder, 2015).

2 Misinformation, covid-19 and hydroxychloroquine

For the scope of this paper, we use the concept of misinformation to define false, distorted, or inaccurate information that is likely to create biased conclusions and misperceptions (Fetzer, 2004; Floridi, 2011; Flynn et al., 2017). We decided to use misinformation instead of disinformation because the difference between the two concepts is intentionality (Jack, 2017; Benkler et al., 2018; Wardle, 2019), and for this study, it does not matter whether the videos we analyze intentionally or not spread misleading information about the use of hydroxychloroquine as treatment of prophylaxis for covid-19.

Misinformation about covid-19 is a global problem and a part of what the World Health Organization defined as “infodemic” (Tangcharoensathien et al., 2020). Misinformation might have had a central role in the spread of the virus in Brazil. As in other populist far-right governments (Bennett & Livingston, 2018), the President of Brazil has been a central agent in the spread of misinformation in political discussions (Alves, 2022) and, particularly, in the polarization of discourses about the pandemic (Ricard & Medeiros, 2020; Soares et al., 2020; Watmough, 2021). Bolsonaro and his supporters regularly promoted unproven drugs to fight covid-19. In particular, they claimed that hydroxychloroquine and chloroquine should be used as treatment and prophylaxis for covid-19, which reverberated on social media (Araujo & Oliveira, 2020; Recuero et al., 2021; Casarões & Magalhães, 2021). Bolsonaro himself promoted the use of hydroxychloroquine for covid-19 in his profiles on Facebook (Soares et al., 2021), Instagram (Soares et al., 2020), and Twitter (Spagnuolo & Orrico, 2020), generating millions of interactions and increasing interest in the topic.

In this context, the Brazilian Ministry of Health used emergency public resources to produce four million chloroquine tablets, which was considered irregular by the Federal Accounts Court. The Brazilian Senate created a Parliamentary Inquiry Commission to investigate the irregularities in the production and promotion of unproven drugs and other illegal actions of Bolsonaro’s government during the pandemic. The investigation identified that pharmaceutical companies paid politicians to promote unproven drugs and doctors to prescribe them, which highly increased the volume of sales of these drugs.

Political authorities, activists, and right-wing alternative media fueled the pro-hydroxychloroquine propaganda on social media (Araujo & Oliveira, 2020; Soares et al., 2020; Recuero et al., 2021). Besides, the mainstream media might have helped to promote misleading information about the topic (Tsfati et al., 2020), as it became a trend in the country and previous studies found that the Brazilian mainstream media play a role in spreading misinformation about covid-19 by merely reproducing false claims from politicians and using clickbait in headlines (Barbosa et al., 2021; Soares & Recuero, 2021). Based on this scenario, we look at videos about hydroxychloroquine and chloroquine on YouTube, in particular those that contain misleading content.

3 YouTube, recommendation system, and echo chambers

Many YouTube channels are sources of false, distorted, or inaccurate information about health (Madathil et al., 2015). This is an issue in the context of the covid-19 pandemic, as videos containing misinformation might fuel the infodemic (Tangcharoensathien et al., 2020). In particular, YouTube’s recommendation system is often criticized, as it might create “rabbit holes” and fuel an information crisis by leading users from one video to another despite how problematic the content of the videos might be (O’Callaghan et al., 2014; Mozilla, 2019). This is rather important because the recommendation system is a major driver of views on YouTube (Zhou et al., 2016). Besides recommended videos, the consumption of political content on YouTube is driven by a combination of several factors, including personal preferences and the influence of content shared on the broader web (Hosseinmardi et al., 2021). The YouTube Team (2019) has already recognized this issue and said they have been working on solutions to improve recommendation algorithms and reduce the spread of harmful content, but recent studies found mixed results, as discussed below.

Many studies found evidence that recommendations from YouTube tend to follow homophily as the algorithms mostly suggest similar content, particularly favoring far-right and radical videos and channels (O’Callaghan et al., 2014; Kaiser & Rauchfleisch, 2020; Röchert et al., 2020). It might ultimately lead to the filter bubble phenomenon (Pariser, 2011), as the algorithms follow homophily patterns and suggest more like-minded content; and to the formation of echo chambers (Sunstein, 2001), as the platform might turn into a space that provides content for users to reinforce their beliefs and potentially become more radicalized.

There are signs of improvement in YouTube’s fight against misinformation since its announcement in 2019, as they acted to reduce anti-vaccine content (O’Donovan, 2019), for example. The comparison between two studies about anti-vaccine content using the same methods shows that anti-vaccine videos reduced from 65% in 2016 (Song & Gruzd, 2017) to 20% in 2019 (Abul-Fottouh et al., 2020). Nevertheless, anti-vaccine videos remained more likely to suggest other anti-vaccine videos (Abul-Fottouh et al., 2020), which indicates that the recommendation algorithm continues following homophily patterns even for harmful content. Furthermore, these studies focused on videos in English. On a related note, many antivaccine channels are still on in Brazil and continue to have their videos monetized on the platform (Tokojima Machado et al., 2020).

YouTube has also made an effort to reduce misinformation about hydroxychloroquine and chloroquine. In April 2021, YouTube announced that they removed videos about unproven drugs for covid-19 that violated YouTube’s content policies (Fonseca, 2021). Nevertheless, previous studies identified a key role of the mainstream media in amplifying misinformation about unproven drugs for covid-19 on YouTube, for example, in videos containing interviews with doctors who defended hydroxychloroquine, chloroquine, and other unproven medicines (Ramos et al., 2020; Barbosa et al., 2021). YouTube’s action to remove pro-hydroxychloroquine content in Brazil estimates the relevance of this topic in the country. This is precisely the reason why we decided to analyze videos about hydroxychloroquine and chloroquine.

4 Research questions and hypotheses

To understand the influence of YouTube’s information ecosystem in the spread of misleading information about unproven drugs in Brazil, we explore how pro-hydroxychloroquine (HCQ) content is propagated on YouTube and the role of YouTube’s recommendation system in promoting this type of content. Therefore, two research questions guide the analysis of this study.

RQ1: How is pro-hydroxychloroquine content propagated on YouTube?

In RQ1, we aim to understand how misleading content about unproven drugs spread on YouTube. In particular, we aim to explore the role of different types of YouTube channels in promoting hydroxychloroquine. Besides, we seek to compare how users interact with pro-HCQ and anti-HCQ videos. Based on this discussion, we propose the following hypotheses:

H1a: Most pro-HCQ videos in our dataset are from mainstream media channels. Previous studies that focused on smaller samples of videos about unproven drugs found that the mainstream media sometimes reverberated misleading content (Ramos et al., 2020; Barbosa et al., 2021). Besides, YouTube has been acting to reduce the amount of misinformation on the platform (YouTube Team, 2019). As data collection was made through YouTube API v3, which is based on YouTube’s search algorithms and recommendation system, we may expect that the recommendation algorithms favor more credible channels.

H1b: Videos containing pro-HCQ content generate more views and interactions (likes, dislikes, comments) compared to anti-HCQ videos. Previous research found that groups engaged in spreading misinformation are often more active (Benkler et al., 2018), including in discussions about unproven drugs for covid-19 on other social media platforms (Recuero et al., 2021; Soares et al., 2021). Therefore, we expect pro-HCQ videos to generate more engagement on YouTube.

We also aim to further explore one particular metadata of our dataset: the list of recommended videos. Thus, our second research question is as follows:

RQ2: How does YouTube’s recommendation system suggest videos about hydroxychloroquine on the platform?

In RQ2, we seek to understand how YouTube’s recommendation algorithms might create echo chambers by favoring homophily. In particular, we aim to explore whether the homophily effect is stronger for pro-HCQ videos compared to anti-HCQ videos. Thus, we propose two hypotheses:

H2a: YouTube’s suggestions follow homophily, possibly fueling the echo chamber effect. That is, pro-HCQ videos are more likely to suggest other pro-HCQ videos, while anti-HCQ videos are more likely to suggest other anti-HCQ videos. As many studies identified, YouTube’s recommendation system tends to suggest similar content, even in videos containing misleading content (Abul-Fottouh et al., 2020; Kaiser & Rauchfleisch, 2020).

H2b: The homophily effect is stronger among pro-HCQ videos. As previous researchers identified, the homophily of YouTube recommendations is particularly strong for radicalized and misleading videos (O’Callaghan et al., 2014; Röchert et al., 2020).

Thus, we expect to find a stronger echo chamber effect for videos containing pro-HCQ content.

5 Methods

We used YouTube Data Tools (Rieder, 2015) to collect videos via the YouTube API v3. We used the Video Network Module to retrieve YouTube videos and recommended videos. The search query used to collect videos was “hidroxicloroquina” OR “cloroquina” (hydroxychloroquine and chloroquine in Portuguese, respectively). We included a language filter, to retrieve videos in Portuguese. This initial search created a seed list of 500 videos. YouTube API v3 provides the seed list based on relevance, that is, these videos were ranked by YouTube as most relevant based on our search query. We also collected the recommended videos (up to 50) for each of the videos in the seed list (crawl depth: 1). YouTube API v3 provides metadata for all collected videos. Data collection was made on March 16, 2021, approximately one year after Bolsonaro started promoting hydroxychloroquine in Brazil.

The resulting dataset contained 11.779 videos. However, most of these videos were not about hydroxychloroquine. Therefore, we excluded videos whose titles did not include the words we used in the search query. We also identified 25 videos in Spanish, nevertheless, we decided to keep these videos in our final dataset because they are linked to videos in Portuguese in the video network based on recommended videos on YouTube. The final dataset comprises 751 videos.

We used mixed methods to explore the dataset. To answer RQ1, we used content analysis (Bardin, 1977) to categorize the videos according to the content, and the channel. To classify the content of each video, we created three categories: pro-HCQ, anti-HCQ, and without a clear position. Table 1 provides a breakdown of how we considered each of these categories.

Table 1
Video content categories framework

Category	Description
Pro-HCQ	We consider that a video contains pro-HCQ content when it reproduces claims that the drugs are a cure or prophylaxis for covid-19, reproduces an interview with someone promoting the drugs, includes stories of people allegedly cured by HCQ, and/or incentivizes people to use the drugs by reverberating protocols from the Brazilian Federal Government or any hospital without mentioning that the drugs do not have efficacy as a treatment for covid-19.
Anti-HCQ	We consider that a video contains anti-HCQ content when it reverberates studies that found no evidence of the efficacy of the drugs for covid-19, reproduces instructions from medical authorities to not use the unproven drugs, includes an interview with someone defending that hydroxychloroquine is an unproven drug for covid-19, and/or clearly mention that despite that some people promote hydroxychloroquine, the drugs do not have efficacy for treating covid-19.
Without a clear position	We consider that a video has no clear position when it does not directly associate the drugs with covid-19, or it does not contain pro-HCQ or anti-HCQ content.

In this classification, we solely considered the position that was held in the video. We did not consider specifically if the video contained misinformation. We understand that even pro-HCQ videos that do not contain false or manipulated information might lead to misperceptions due to how the discussion was framed in society – as a political discussion rather than a scientific one (Recuero et al., 2021). Furthermore, by using pro and anti HCQ categories, we follow the same pattern of categories as studies that focused on content about vaccines on YouTube (Song & Gruzd, 2017; Abul-Fottouh et al., 2020), which contributes to the reproducibility of our method.

To classify the YouTube channels, we considered six categories. These categories emerged from a pre-analysis of the data (Bardin, 1977). To classify each channel, we considered its name, description, and videos. We describe each category in Table 2.

Table 2
YouTube channel categories framework

Category	Description
Mainstream media	YouTube channels from free-to-air television channels, cable television channels, daily newspapers, radio stations, traditional online news portals, and mainstream media in general.
Alternative media	YouTube channels from alternative sources of information, such as hyperpartisan media, independent media, and other types of nontraditional news outlets.
Political	YouTube channels in which the main topic is politics, such as channels from politicians, activists, political analysts, and journalists covering political issues.
Entertainment	YouTube channels in which the main topic is entertainment. For example, music, humor, cinema, etc.
Science, health, and education	YouTube channels from organizations associated with research, health, and/ or education in general, or YouTube channels in which the main topic is science, health, and/or education.
Other	YouTube channels that cannot be classified into any other category.

Three independent coders classified the data. The coding was performed by the authors of this study for better accuracy and to allow us to have more in-depth insights into our data. Each coder watched all the 751 videos to classify the content and the channel following the framework presented above. In case videos contained both pro and anti HCQ content, coders were instructed to identify which position stands out in the video. To ensure the reliability of the coding, we calculated Cohen’s Kappa. Table 3 provides detailed information about the interrater agreement. We reached a substantial agreement for the content of the videos and an almost perfect agreement for the channels (Cohen, 1960; Landis & Koch, 1977).

Table 3
Information about interrater agreement

Coders	Content	Channel
Coders 1 and 2	κ = .73	κ = .901
Coders 1 and 3	κ = .746	κ = .931
Coders 2 and 3	κ = .774	κ = .921
Overall	κ = .75	κ = .917

We considered the agreement between at least two coders for the final classification. There were disagreements between the three coders about the content of six videos. We decided to classify the content of these videos without a clear position for the final classification.

We used statistical tests to explore the metadata of the videos provided by YouTube’s API. We tested the data (views, comments, likes, dislikes) for normality and homogeneity of variance and identified that neither could be assumed. Besides, sample sizes were not equal. Therefore, we used Welch’s t-test to correct these violations (Zimmerman, 2004) and test for statistical differences in means of the metadata from anti-HCQ and pro-HCQ videos. There were missing values in some videos, therefore we did not consider these videos in the tests. We used JASP (JASP Team, 2020) to perform the statistical tests and create the descriptive plots presented in this paper.

To answer RQ2, we relied on social network analysis (Wasserman & Faust, 1994). We used the Video Network Module to collect videos via YouTube Data Tolls (Rieder, 2015). Thus, we retrieved a network of recommended videos, in which nodes are YouTube videos and edges represent YouTube’s recommendation of videos. Based on the content analysis, we focused on two metrics to analyze how YouTube’s recommendation algorithm might fuel the emergence of echo chambers.

We used the E-I index to compare the connections within groups and between groups (Krackhardt & Stern, 1988). Usually, the E-I index ignores the direction of ties. However, as we are looking at recommended videos, we only considered ties that nodes send in the network. That is, we only look at which videos are recommended by each video in the network, based on the content of the videos (content analysis) to define whether a tie is internal or external. E-I index ranges from -1 (all ties are internal to the group) to +1 (all ties are external to the group). Therefore, a negative index indicates that connections within groups are more frequent than connections between groups, while a positive index indicates the opposite.

We also used the clustering coefficient metric (Watts & Strogatz, 1998) to explore the tendency of nodes from each group to cluster together. The clustering coefficient metric measures how close the neighbors of a particular node are, that is, how interconnected the connections of a node are. The average clustering coefficient of groups provides information about the density of the connections within the group. In the case of our analysis, this is helpful to further explore the likelihood of someone who watches a pro-HCQ video, for example, to continue watching more pro-HCQ videos – the echo chamber (Sunstein, 2001) or “rabbit hole” (O’Callaghan et al., 2014) effect.

Finally, we used once again Welch’s t-test to compare the indegree from anti-HCQ and pro-HCQ videos. We used indegree because this metric is related to how often videos from our dataset were recommended by YouTube. This test was used to understand how YouTube’s recommendation system could be favoring videos with a particular type of content in the debate about unproven drugs.

6 Results

Our first research question focused on the content of the videos in our dataset. Out of the 751 videos we analyzed, most contained anti-HCQ content (368 – 49%), closely followed by pro-HCQ videos (323 – 43%). Videos without a clear position about the use of hydroxychloroquine as a treatment or prophylaxis for covid-19 were the minority in our dataset (60 – 8%). Figure 1 provides a breakdown of the categories we analyzed (content and type of channel).

Figure 1
Type of content per type of channel

As described in figure 1, videos from the mainstream media were the majority in all types of content. Mainstream media account for 72% of the videos containing pro-HCQ content and 52% of antiHCQ videos. Videos from alternative media channels and science, health and education channels more often contained anti-HCQ content, frequently reverberating scientific tests with the drugs.

Many of the mainstream media videos containing pro-HCQ content are from Brazilian free-to-air television channels. These videos reproduced parts of programs broadcasted by these free-to-air television channels. Other mainstream media videos containing proHCQ content reproduced part of speeches, Facebook lives, or interviews from Bolsonaro, in which he promotes the use of hydroxychloroquine. In these videos, there are no mentions of reliable sources that consider hydroxychloroquine and chloroquine as unproven drugs for covid-19.

These findings support our Hypothesis 1a: Most pro-HCQ videos in our dataset are from mainstream media channels. This result points to the role of mainstream media in amplifying misinformation (Tsfati et al., 2020) by posting YouTube videos containing pro-HCQ content. This result is also in line with previous analysis that also found that the Brazilian mainstream media played a role in promoting unproven drugs (Ramos et al., 2020; Barbosa et al., 2021). Likely, this result was partially influenced by how we collected the data. We used YouTube’s API which provides videos based on YouTube’s search system. Therefore, videos from mainstream media might be more likely to be suggested in searches on YouTube. Nevertheless, we consider this finding relevant because it points to the role of mainstream media in fueling misinformation about unproven drugs.

We used video metadata provided by YouTube’s API to explore how different numbers of views and interactions were between pro-HCQ and anti-HCQ videos. We compared four elements: views, comments, likes, and dislikes. As explained in the methods section, we used Welch’s t-test to correct violations of normality and homogeneity of variance (Zimmerman, 2004).

We found statistical significant differences for views, Welch’s t(655.85)= 3.256, p = .001, 95% CI [0.1, 0.4]; comments, Welch’s t(596.43)= 3.031, p = .003, 95% CI [0.08, 0.39]; and likes, Welch’s t(526.5)= 2.468, p = .014, 95% CI [0.04, 0.34]. Pro-HCQ videos had higher numbers for all these interaction metrics. On the other hand, Anti-HCQ videos received more dislikes, but the difference was not statistically significant, Welch’s t(542.7)= -1.872, p = .062, 95% CI [-0.29, 0.01]. Table 4 and figures 2-5 provide additional information about the tests.

Table 4
Views and interactions in pro-HCQ and anti-HCQ videos

Metadata	Content	N	Mean	SD
Views	Pro-HCQ	323	61854.94	111381.88
	Anti-HCQ	368	35345.17	101270.11
Comments	Pro-HCQ	321	469.22	760.80
	Anti-HCQ	346	309.64	578.81
Likes	Pro-HCQ	319	4732.07	11971.50
	Anti-HCQ	367	2806.09	7656.61
Dislikes	Pro-HCQ	319	131.10	214.88
	Anti-HCQ	367	180.21	446.48

Figure 2
Mean number of views

Figure 3
Mean number of comments

Figure 4
Mean number of likes

Figure 5
Mean number of dislikes

We also looked at specific mainstream media channels that posted both pro-HCQ videos and anti-HCQ videos. This exploratory analysis aimed to further investigate whether the larger number of views, comments, and likes for pro-HCQ videos was related to the content of the videos or was a consequence of channels with more subscriptions sharing pro-HCQ content. We selected the four channels with the most videos in our dataset among those that posted proHCQ and anti-HCQ videos. We compared views and interactions of pro-HCQ and anti-HCQ videos in these four channels using Welch’s t-test. Table 5 provides a breakdown of the statistical tests.

Table 5
Comparison of channels that posted both pro-HCQ and antiHCQ videos

Channel	Variable	Content	N	Mean	SD	Welch’s t	df	P
UOL	Views	Pro	40	38123.33	61256.71	2.64	47.15	.011
		Anti	14	11019.79	12814.77
	Comments	Pro	40	330.83	418.7	0.15	18.51	.880
		Anti	14	306	552.12
	Dislikes	Pro	40	130.28	165.62	-0.2	17.44	.843
		Anti	14	144.36	242.45
	Likes	Pro	40	791.68	1489.43	2.16	44.89	.036
		Anti	14	261.93	255.65
Band Jornalismo	Views	Pro	18	51080.61	71598.9	1.77	27.27	.087
		Anti	17	16402	40794.4
	Comments	Pro	18	308.61	253.32	0.16	23.85	.873
		Anti	17	287.41	483.98
	Dislikes	Pro	18	109.72	107.33	0.1	22.29	.919
		Anti	17	103.41	231.48
	Likes	Pro	18	1340.44	1943.89	2.1	20.3	.048
		Anti	17	331.53	592.17
Jovem Pan News	Views	Pro	17	76162.29	142222.76	1.96	16.36	.067
		Anti	14	8040.93	13672.26
	Comments	Pro	17	420.12	833.83	1.26	18.78	.224
		Anti	14	154.57	225.38
	Dislikes	Pro	17	127.94	168.65	-0.2	23.14	.843
		Anti	14	142.86	233.01
	Likes	Pro	17	4130.77	7386.77	2.16	16.15	.046
		Anti	14	251.79	457.64
Jornal da Record	Views	Pro	21	49721.05	87808.18	2.29	20.2	.033
		Anti	6	5833.33	3311.35
	Comments	Pro	21	259.43	370.51	2.19	24.34	.038
		Anti	6	69	77.89
	Dislikes	Pro	21	70.91	94.48	2.59	21.75	.017
		Anti	6	16.33	11.02
	Likes	Pro	21	1891.19	3388.12	2.32	20.21	.031
		Anti	6	172.83	130.9

In all four channels, pro-HCQ videos received more views compared to anti-HCQ videos, although this result was statistically significant in only two cases. In all the cases, pro-HCQ videos received more likes, all four with statistical significance. In all cases, pro-HCQ videos received more comments, but only in one with statistical significance. Finally, in two cases anti-HCQ videos received more dislikes, none with statistical significance, and in the other two cases, pro-HCQ videos received more dislikes, one with statistical significance. In general, these results, especially those related to views and likes, indicate that pro-HCQ videos tend to receive more views and interactions because of the content of the video, as we identified relevant differences in videos with different content in the same channel.

These findings mostly support our hypothesis 1b: Videos containing pro-HCQ content generate more views and interactions (likes, dislikes, comments) compared to anti-HCQ videos. Interestingly, the only metric that the mean in pro-HCQ videos is not statistically significantly higher than anti-HCQ is dislikes, which can be considered a negative type of interaction. These results indicate that YouTube (and possibly other social media) users that supported the use of hydroxychloroquine and chloroquine for covid-19 were likely engaged in promoting pro-HCQ videos from our dataset, as they received more views, likes, and comments. These findings are in line with previous studies that identified that social media users engaged in spreading misinformation are often more active (Benkler et al., 2018), even in discussions about unproven drugs for covid-19 (Recuero et al., 2021; Soares et al., 2021).

To explore our second research question, we focused on the recommended videos’ metadata. As detailed in the methods section, we used social network analysis to explore how YouTube’s recommendation system could create “echo chambers” by the following homophily when suggesting related videos. The network was composed of 751 nodes and 10.116 edges.

We examined three metrics to explore hypotheses 2a and 2b. First, we calculated the E-I index (table 6), particularly by looking at the connections made by the nodes. That is, we analyzed whether videos recommended other videos containing the same content or different content. We found that the E-I index was very close to zero (-0.02), which indicates a similar number of internal and external ties. We discovered that pro-HCQ videos were more likely to recommend other pro-HCQ videos. On the other hand, anti-HCQ videos were more likely to create external connections, that is, to recommend pro-HCQ videos and videos without a clear position. Finally, videos without a clear position were also more likely to create external connections, but this result is influenced by the small number of videos in this category – only 8% of our dataset.

Table 6
Metrics used to explore homophily and clustering

Group	E-I index	Average clustering coefficient
Anti-HCQ	0.06	0.132
Pro-HCQ	-0.22	0.178
Without a clear position	0.75	0.156
Network	-0.02	0.157

We also calculated the average clustering coefficient for the entire network and for each group to explore how interconnected the videos from our dataset were (table 6). The results follow the same tendency as the E-I index. The average clustering coefficient of pro-HCQ videos is higher than other groups and the entire network. This indicates higher density within the pro-HCQ group, that is, the neighborhood of videos is more closely interconnected. The average clustering coefficient of anti-HCQ videos and videos without a clear position group is smaller than the entire network, which indicates a lower tendency for interconnections. A breakdown of the E-I index and clustering coefficient is provided in table 6.

These results indicate that pro-HCQ are more likely to create an echo chamber (Sunstein, 2001) effect and lead YouTube users into “rabbit holes” (O’Callaghan et al., 2014), even though this tendency was weaker than initially expected. As the recommendation system tends to be a driver of views on YouTube (Zhou et al., 2016), users that watch a video containing pro-HCQ content are more likely to watch other videos promoting unproven drugs for covid-19.

We also decided to explore how often videos from our dataset were recommended by YouTube. In particular, we wanted to understand whether pro-HCQ or anti-HCQ videos were more likely to be recommended. Therefore, we compared the indegree of these two types of videos using Welch’s t-test. We identified that YouTube was more likely to recommend pro-HCQ videos in our dataset. The result of the test was statistically significant, Welch’s t(579.54) = 2.543, p = .011, 95% CI [0.05, 0.35]. Additional information about the comparison between the two groups is provided in table 7 and figure 6.

Table 7
Indegree of pro-HCQ and anti-HCQ videos

Group	N	Mean	SD
Pro-HCQ	323	16.51	32.05
Anti-HCQ	368	11.03	23.25

Figure 6
Mean number of indegree

As we previously identified that pro-HCQ videos received more views, we decided to further explore how views could have influenced indegree (number of recommendations in our dataset). That is, we wanted to understand if there was a correlation between popularity (number of views) and recommendations on YouTube. Therefore, we calculated the Pearson coefficient to measure the correlation between views and indegree. Although there was a positive correlation, this was weak: Pearson’s r(689) = 0.291, p < .001. This indicates that there was some relationship between popularity and recommendations, however, the relationship was unsubstantial to explain why pro-HCQ videos were more recommended in our dataset. Figure 7 provides visual data about the relationship between views and indegree.

Figure 7
Correlation between views and indegree

These findings suggest that pro-HCQ videos were more likely to recommend other pro-HCQ videos and that this tendency was fueled by YouTube’s recommendation algorithms. Consequently, this finding highlights the role of YouTube in fueling misinformation about unproven drugs for covid-19.

Our results do not support Hypothesis 2a: YouTube’s suggestions follow homophily, possibly fueling the echo chamber effect. On the other hand, our findings support Hypothesis 2b: The homophily effect is stronger among pro-HCQ videos. These results are in line with other studies that identified YouTube’s algorithms’ tendency to follow homophily in recommending videos containing misleading content (O’Callaghan et al., 2014; Abul-Fottouh et al., 2020; Kaiser & Rauchfleisch, 2020; Röchert et al., 2020).

7 Discussion

Based on the results of our analysis, we identified that the Brazilian mainstream media and YouTube played key roles in promoting unproven drugs for covid-19 in Brazil. As we explored in our first research question, most videos containing pro-HCQ content were posted by mainstream media channels. An important note is that many of these videos contain parts of programs broadcasted by free-to-air television channels, which indicates the existence of a larger ecosystem of misinformation. This is relevant because over half of Brazilians mentioned television channels as a source for getting news in the latest Reuters Digital News Report (Newman et al., 2022).

Some of the channels that posted videos containing pro-HCQ videos are associated with brands among the top five most trusted in Brazil (Newman et al., 2022), such as Band News (35 pro-HCQ videos and trusted by 61% of the Brazilians interviewed in the latest Reuters Digital News Report), Record News (51 and 61%), and UOL (40 and 57%). Band and Record News are both free-to-air television channels. Many of these videos include journalists or TV hosts that promote hydroxychloroquine and chloroquine, and interviews with politicians and health specialists that defend the unproven drugs.

Jovem Pan, the largest radio station in Brazil, also posted 46 videos containing pro-HCQ content. They are similar to the videos posted by Band and Record News, often containing hosts or guests promoting the drugs. UOL is the largest online news portal in Brazil. Videos posted by UOL differ from the others, as many of them simply reproduce parts of speeches, Facebook lives, or interviews from Bolsonaro in which he promotes the unproven drugs. These videos do not contain any corrections over Bolsonaro’s false claims nor mention scientific studies that did not identify any efficacy of the use of these drugs in the treatment of covid-19.

The role of mainstream media in promoting unproven drugs for covid-19 is particularly relevant because of how Brazilians trust these sources. The mainstream media might fuel misinformation spread even when correcting false claims (Tsfati et al., 2020). This turns out to be even more harmful when the mainstream media is in fact promoting misleading content. When the mainstream media produces misleading content, social media users often share this content to reinforce misinformation narratives (Soares & Recuero, 2021). The same is potentially true for YouTube videos promoting unproven drugs, which could end up fueling the so-called infodemic (Tangcharoensathien et al., 2020).

Another important factor in the spread of pro-HCQ content on YouTube is the role of users and how they interact with these videos. We identified that videos containing pro-HCQ content received more likes, comments, and views than videos containing anti-HCQ content. These findings suggest that YouTube users were more active in watching and interacting with pro-HCQ videos. Furthermore, these findings might be the result of more activity on other social media platforms and messaging apps (Hosseinmardi et al., 2021). Possibly, these videos were shared in other spaces to promote hydroxychloroquine and chloroquine for covid-19, which could explain the higher number of views, likes, and comments (and lower number of dislikes).

Finally, we identified that YouTube played a key role in the process. The higher number of views and interactions in proHCQ videos might have been partially influenced by how YouTube recommended these videos. We identified that YouTube was more likely to suggest pro-HCQ videos than anti-HCQ videos. Therefore, YouTube’s recommendation algorithms end up promoting unproven drugs. On the other hand, we also identified that the correlation between views and indegree was weak. This indicates that the higher number of views in videos containing pro-HCQ content might be the result of personal preferences and the promotion of misleading content on other social media platforms, as suggested by Hosseinmardi et al. (2021).

In addition, we identified that the videos suggested when a user watched a pro-HCQ video followed homophily patterns to some extent. This could reinforce the echo chamber (Sunstein, 2001) effect, in which users who watched a video promoting unproven drugs would likely watch more videos promoting these drugs. Therefore, YouTube users could go into “rabbit holes” of misleading information (O’Callaghan et al., 2014). This is particularly relevant because YouTube’s recommendation system might be a major driver of views (Zhou et al., 2016).

YouTube Team (2019) says they have been working to improve recommendation algorithms. This action might explain the large number of videos from mainstream media channels in our dataset. Nevertheless, we identified that YouTube’s recommendation algorithms are contributing to the spread of misleading information, in particular by often suggesting videos containing pro-HCQ content. Another factor that might influence this issue is language. Most of the videos in our dataset are in Portuguese. YouTube may have been developing ways to improve recommendation algorithms for videos in English, but there are still many flaws regarding videos in Portuguese – and possibly in other languages as well.

In April 2021, YouTube announced that they removed thousands of videos about unproven drugs. As we collected our dataset in March 2021, we revised it to identify which videos were removed. As of August 6, 2021, only 21 videos were removed because the content violated YouTube’s policies. Most of these videos (20 – 95%) contained pro-HCQ content. As for the type of channel, the majority were posted by mainstream media channels (15 – 71%). Also, 18 videos were unavailable or private, of which 12 (67%) contained pro-HCQ content. The majority of the unavailable or private videos were posted by political channels (13 – 72%). Despite YouTube’s actions to reduce the number of videos promoting unproven drugs on the platform, 90% of the videos containing pro-HCQ content from our dataset were still available. The removed, unavailable or private pro-HCQ videos received over four million views by March 2021, when we collected the data. In particular, eight videos posted by Alexandre Garcia, a political commentator, and Bolsonaro’s supporter, were unavailable or private – altogether, they accounted for almost two million views. This indicates that YouTube’s actions to remove misinformation about unproven drugs had little effect, as most pro-HCQ videos were still online and those that were not available received millions of views before they were removed or made private/unavailable.

A key implication of our findings is the necessity to look at the role of legitimate actors in spreading misinformation. Studies analyzing misinformation spread on social media often focus on hyperpartisan outlets, coordinated behavior, bots, and “bad actors” in general. While it is relevant to analyze the role of bad actors, our findings suggest that is also important to look at the role of authentic actors (such as the mainstream media) in the misinformation spread. This is rather important because bad actors are often related to specific (usually radicalized) groups, but mainstream media outlets reach a much larger audience. Therefore, future studies ought to look at the impact of authentic actors on misinformation spread.

This study has some limitations. We used YouTube API v3 to collect data. This API uses YouTube’s search rank to provide videos based on the search query we used. Therefore, this might have influenced the large number of videos from mainstream media channels. Besides, our sample is not representative of the entirety of YouTube’s content and we are unable to measure how many other videos about hydroxychloroquine and chloroquine are available on YouTube. There are also some limitations in the use of the E-I index to measure homophily. Although this metric helped us to identify a tendency of pro-HCQ videos to recommend more pro-HCQ videos, the formula used for the calculation of the E-I index is not a robust statistical test.

8 Conclusion

In this study, we focused on how YouTube videos contributed to the “infodemic” (Tangcharoensathien et al., 2020). In particular, we looked at videos about hydroxychloroquine and chloroquine, two unproven drugs promoted by Brazilian President Jair Bolsonaro and his supporters. We used mixed methods to explore two research questions. RQ1: How is pro-hydroxychloroquine content propagated on YouTube? RQ2: How does YouTube’s recommendation system suggest videos about hydroxychloroquine on the platform?

We identified that Brazilian mainstream media played a key role in promoting these unproven drugs, as most videos containing pro-HCQ content were posted by mainstream media channels. Many of these videos reproduced content from free-to-air television channels, which suggests the existence of a larger ecosystem of misinformation about unproven drugs for covid-19 in Brazil. Furthermore, we identified that pro-HCQ videos received more views, likes, and comments, which suggests that YouTube users interacted more with videos promoting the drugs.

We also found that YouTube’s recommendation system favored pro-HCQ content on the platform. We identified that pro-HCQ videos were more likely to recommend other pro-HCQ videos, possibly creating “rabbit holes” (O’Callaghan et al., 2014) and reinforcing an “echo chamber” (Sunstein, 2001) effect. Furthermore, we identified that YouTube’s algorithms more often recommended pro-HCQ videos than anti-HCQ videos. Therefore, YouTube ended up promoting proHCQ content and fueled the spread of misinformation.

One of our study contributions is the analysis of videos in Portuguese, since videos in English are the focus of most of the research on health misinformation on YouTube (Abul-Fottouh et al., 2020). The discussion on the roles of the Brazilian mainstream media and YouTube in fueling misinformation about unproven drugs for covid-19 is another contribution of our study. The finding on the role of the mainstream media in amplifying misinformation about unproven drugs is relevant to exploring how legitimate actors might contribute to the covid-19 infodemic. Besides, YouTube’s algorithms are likely to play a similar role in fueling misinformation spread in other countries as well as we identified in our study. This tendency might be stronger among nonEnglish videos, which is something that future studies can explore.

REFERENCES

• Abul-Fottouh, D., Song, M. Y., & Gruzd, A. (2020). Examining algorithmic biases in YouTube’s recommendations of vaccine videos. International Journal of Medical Informatics, 140(104175), 1–6 . DOI: 10.1016/j.ijmedinf.2020.104175
  » https://doi.org/10.1016/j.ijmedinf.2020.104175
• Alves, M. (2022). As flutuações de longo prazo da polarização no Brasil - análise de compartilhamento de informações políticas entre 2011 e 2019. Revista Dados, 66(2), 1–45. DOI: 10.1590/dados.2023.66.2.287
  » https://doi.org/10.1590/dados.2023.66.2.287
• Araújo, R., & Oliveira, T. A. (2020). Desinformação e mensagens sobre a hidroxicloroquina no Twitter: da pressão política à disputa científica. Atoz: novas práticas em informação e conhecimento, 9(2), 196–205. DOI: 10.5380/atoz.v9i2.75929
  » https://doi.org/10.5380/atoz.v9i2.75929
• Barbosa, B., Fávero, B., Ely, D., & Barbosa, J. (2021, February 26). Impulsionados pela imprensa, médicos que desinformam sobre Covid-19 somam 30 mi de visualizações no YouTube Radar Aos Fatos. Retrieved from www.aosfatos.org/noticias/impulsionados-pelaimprensa-medicos-que-desinformam-sobre-covid-19-somam-30-mide-visualizacoes-no-youtube/
  » www.aosfatos.org/noticias/impulsionados-pelaimprensa-medicos-que-desinformam-sobre-covid-19-somam-30-mide-visualizacoes-no-youtube/
• Bardin, L. (1977). Análise de conteúdo PUF.
• Benkler, Y., Faris, R., & Roberts, H. (2018). Network Propaganda: Manipulation, disinformation, and radicalization in American politics Oxford University Press.
• Bennett, W., & Livingston, S. (2018). The disinformation order: Disruptive communication and the decline of democratic institutions. European Journal of Communication, 33(2), 122–139. DOI: 10.1177/0267323118760317
  » https://doi.org/10.1177/0267323118760317
• Casarões, G., Magalhães, D. (2021). The hydroxychloroquine alliance: how far-right leaders and alt-science preachers came together to promote a miracle drug. Revista de Administração Pública, 55(1), 197–214. DOI: 10.1590/0034-761220200556
  » https://doi.org/10.1590/0034-761220200556
• Cohen, J. A. (1960). A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement, 20(1), 37–46. DOI: 10.1177/001316446002000104
  » https://doi.org/10.1177/001316446002000104
• Fetzer, J. H. (2004). Disinformation: The Use of False Information. Minds and Machines, 14(2), 231–240. DOI: 10.1023/B:MIND.0000021683.28604.5b
  » https://doi.org/10.1023/B:MIND.0000021683.28604.5b
• Floridi, L. (2011). The philosophy of information Oxford University Press.
• Flynn, D. J., Nyhan, B., & Reifler, J. (2017). The Nature and Origins of Misperceptions: Understanding False and Unsupported Beliefs About Politics. Advances in Political Psychology, 38(1), 127–150. DOI: 10.1111/pops.12394
  » https://doi.org/10.1111/pops.12394
• Fonseca, N. (2021, April 16). YouTube remove vídeos que indicam o uso de cloroquina e ivermectina contra Covid CNN Brasil. Retrieved from www.cnnbrasil.com.br/tecnologia/youtube-remove-videosque-indicam-o-uso-de-cloroquina-e-ivermectina-contra-covid/
  » www.cnnbrasil.com.br/tecnologia/youtube-remove-videosque-indicam-o-uso-de-cloroquina-e-ivermectina-contra-covid/
• Hosseinmardi, H., Ghasemian, A., Clauset, A., Mobius, M., Rothschild, D. M., & Watts, D. J. (2021). Examining the consumption of radical content on YouTube. Proceedings of the National Academy of Sciences, 118(32). DOI: 10.1073/pnas.2101967118
  » https://doi.org/10.1073/pnas.2101967118
• Jack, C. (2017). Lexicon of Lies: Terms for Problematic Information Data & Society Research Institute.
• JASP Team. (2020). JASP (Version 0.14.1) [computer software].
• Kaiser, J., & Rauchfleisch, A. (2020). Birds of a Feather Get Recommended Together: Algorithmic Homophily in YouTube’s Channel Recommendations in the United States and Germany. Social Media + Society, 6(4), 1–15. DOI: 10.1177/2056305120969914
  » https://doi.org/10.1177/2056305120969914
• Krackhardt, D., & Stern, R. N. (1988). Informal Networks and Organizational Crises: An Experimental Simulation. Social Psychology Quarterly, 51(2), 123–140. DOI: 10.2307/2786835
  » https://doi.org/10.2307/2786835
• Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. DOI: 10.2307/2529310
  » https://doi.org/10.2307/2529310
• Machado, C. C. V., Dourado, D. A., Santos, J. G., & Santos, N. (2020). Ciência Contaminada: Analisando o contágio de desinformação sobre coronavírus via YouTube Laut. Retrieved from https://laut.org.br/ciencia-contaminada.pdf
  » https://laut.org.br/ciencia-contaminada.pdf
• Madathil, K. C., Rivera-Rodriguez, A. J., Greenstein, J. S, & Gramopadhye, A K. (2015). Healthcare information on YouTube: A systematic review. Health Informatics Journal, 21(3), 173–194. DOI: 10.1177/1460458213512220
  » https://doi.org/10.1177/1460458213512220
• Mozilla. (2019, n.d.). How healthy is the internet? Retrieved from https://internethealthreport.org/2019/
  » https://internethealthreport.org/2019/
• Newman, N., Fletcher, R., Robertson, C. T., Eddy, K., & Nielsen, R. K. (2022). Reuters Institute Digital News Report 2022 Reuters Institute, Oxford University.
• O’Callaghan, D., Greene, D., Conway, M., Carthy, J., & Cunningham, P. (2014). Down the (White) Rabbit Hole: The Extreme Right and Online Recommender Systems. Social Science Computer Review, 33(4), 459–478. DOI: 10.1177/0894439314555329
  » https://doi.org/10.1177/0894439314555329
• O’Donovan, C. (2019, February 22). YouTube Just Demonetized Anti-Vax Channels BuzzFeed News. Retrieved from www.buzzfeednews.com/ article/carolineodonovan/youtube-just-demonetized-anti-vax-channels
  » www.buzzfeednews.com/ article/carolineodonovan/youtube-just-demonetized-anti-vax-channels
• Pariser, E. (2011). The filter bubble: What the Internet is hiding from you Penguin.
• Ramos, T. B., Bokehi, L. C., Bokehi, R. C., Pinheiro, T. S., Oliveira, E. B., Torres, R. S. G., Bokehi, J. R., Calil-Elias, S., & Castilho, S. R. (2020). YouTube as a source of information on chloroquine and hydroxychloroquine during the COVID-19 pandemic. Journal of Science Communication, 19(07), 1–19. DOI: 10.22323/2.19070206
  » https://doi.org/10.22323/2.19070206
• Recuero, R., Soares, F. B., & Zago, G. (2021). Polarização, Hiperpartidarismo e Câmaras de Eco: Como circula a Desinformação sobre Covid-19 no Twitter. Contracampo, 40(1), 1–17. DOI: 10.22409/contracampo.v40i1.45611
  » https://doi.org/10.22409/contracampo.v40i1.45611
• Ricard, J., & Medeiros, J. (2020). Using misinformation as a politic al weapon: COVID 19 and Bolsonaro in Brazil. Harvard Kennedy School (HKS) Misinformation Review, 1(2), 1-6. DOI: 10.37016/mr-2020-013
  » https://doi.org/10.37016/mr-2020-013
• Rieder, B. (2015). YouTube Data Tools (Version 1.22) [software]. Available from https://tools.digitalmethods.net/netvizz/youtube/
  » https://tools.digitalmethods.net/netvizz/youtube/
• Röchert, D., Weitzel, M., & Ross, B. (2020). The homogeneity of rightwing populist and radical content in YouTube recommendations. Proceedings of the International Conference on Social Media and Society Association for Computing Machinery. DOI: 10.1145/3400806.3400835
  » https://doi.org/10.1145/3400806.3400835
• Song, M. Y., & Gruzd, A. (2017). Examining Sentiments and Popularity of Proand Anti-Vaccination Videos on YouTube. Proceedings of the 8th International Conference on Social Media & Society Association for Computing Machinery. DOI: 10.1145/3097286.3097303
  » https://doi.org/10.1145/3097286.3097303
• Soares, F. B., Recuero, R., & Zago, G. (2019). Asymmetric Polarization on Twitter and the 2018 Brazilian Presidential Elections. Proceedings of the 10th International Conference on Social Media and Society Association for Computing Machinery. DOI: 10.1145/3328529.3328546
  » https://doi.org/10.1145/3328529.3328546
• Soares, F. B., Bonoto, C., Viegas, P., Salgueiro, I., & Recuero, R. (2020). Disputas discursivas e desinformação no Instagram sobre o uso da hidroxicloroquina como tratamento para o Covid-19. Proceedings of the Congresso Brasileiro de Ciências da Comunicação UFBA. Retrieved from https://portalintercom.org.br/anais/nacional2020/resumos/R15-0550-1.pdf
  » https://portalintercom.org.br/anais/nacional2020/resumos/R15-0550-1.pdf
• Soares, F. B., & Recuero, R. (2021). How the Mainstream Media Help to Spread Disinformation about Covid-19. M/C Journal, 24(1). DOI: 10.5204/mcj.2735
  » https://doi.org/10.5204/mcj.2735
• Soares, F. B., Viegas, P., Bonoto, C., & Recuero, R. (2021). Covid-19, desinformação e Facebook: circulação de URLs sobre a hidroxicloroquina em páginas e grupos públicos. Galáxia, 46(46), 1–24. DOI: 10.1590/1982-2553202151423
  » https://doi.org/10.1590/1982-2553202151423
• Spagnuolo, S., & Orrico, A. (2020, March 2023). Coronavírus: interesse sobre cloroquina disparou após vídeo de Bolsonaro Núcleo. Retrieved from https://nucleo.jor.br/redes/2020-03-23-cloroquina-bolsonarovideo-coronavirus
  » https://nucleo.jor.br/redes/2020-03-23-cloroquina-bolsonarovideo-coronavirus
• Sunstein, C. R. (2001). Republic.com Princeton University Press.
• Tangcharoensathien, V., Calleja, N., Nguyen, T., Purnat, T., D’Agostino, M., Garcia-Saiso, S., Landry, M., Rashidian, A., Hamilton, C., AbdAllah, A., Ghiga, I., Hill, A., Hougendobler, D., van Andel, J., Nunn, M., Brooks, I., Sacco, P. L., De Domenico, M., Mai, P., Gruzd, A., Alaphilippe, A., & Briand, S. (2020). Framework for Managing the COVID-19 Infodemic: Methods and Results of an Online Crowdsourced WHO Technical Consultation. J Med Internet Res, 22(6). DOI: 10.2196/19659
  » https://doi.org/10.2196/19659
• Tokojima Machado, D. F., de Siqueira, A. F., & Gitahy, L. (2020). Natural Stings: Selling Distrust About Vaccines on Brazilian YouTube. Frontiers in Communication, 5, 2–9. DOI: 10.3389/fcomm.2020.577941
  » https://doi.org/10.3389/fcomm.2020.577941
• Tsfati, Y., Boomgaarden, H. G., Strömbäck, J., Vliegenthart, R., Damstra A., & Lindgren, E. (2020). Causes and consequences of mainstream media dissemination of fake news: literature review and synthesis. Annals of the International Communication Association, 44(2), 157–173. DOI: 10.1080/23808985.2020.1759443
  » https://doi.org/10.1080/23808985.2020.1759443
• Wardle, C. (2019). Understanding Information Disorder First Draft.
• Wasserman, S., & Faust, K. (1994). Social Network Analysis Cambridge University Press.
• Watmough, S. P. (2021). Jair Bolsonaro: Far-Right Firebrand and Cheerleader for Dictatorship European Center for Populism Studies. Retrieved from www.populismstudies.org/jair-bolsonaro-far-rightfirebrand-and-cheerleader-for-dictatorship/
  » www.populismstudies.org/jair-bolsonaro-far-rightfirebrand-and-cheerleader-for-dictatorship/
• Watts, D. J., & Strogatz, S. H. (1998). Collective dynamics of ‘smallworld’ networks. Nature, 393, 440-442. DOI:10.1038/30918
  » https://doi.org/10.1038/30918
• YouTube Team. (2019, January 25). Continuing our work to improve recommendations on YouTube YouTube Official Blog. Retrieved from https://blog.youtube/news-and-events/continuing-our-work-to-improve
  » https://blog.youtube/news-and-events/continuing-our-work-to-improve
• Zhou, R., Khemmarat, S., Gao, L., Wan, J., & Zhang, J. (2016). How YouTube videos are discovered and its impact on video views. Multimed Tools Appl, 75, 6035–6058. DOI: 10.1007/s11042-015-3206-0
  » https://doi.org/10.1007/s11042-015-3206-0
• Zimmerman, D. W. (2004). A note on preliminary tests of equality of variances. The British Journal of Mathematical and Statistical Psychology, 57, 173–181. DOI: 10.1348/000711004849222
  » https://doi.org/10.1348/000711004849222

Edited by

Desk Review Editor: Kérley Winques

Publication Dates

• Publication in this collection
  13 Feb 2023
• Date of issue
  Sep-Dec 2022

History

• Received
  01 July 2022
• Reviewed
  23 July 2022
• Reviewed
  08 Oct 2022
• Accepted
  16 Oct 2022