Transparency and Replication in Brazilian Political Science: A First Look^*

Transparência e Replicação na Ciência Política Brasileira: Um Primeiro Olhar

Transparencia y Replicación en la Ciencia Política Brasileña: Una Primera Mirada

Transparence et Réplication dans la Science Politique Brésilienne: Premier Regard

ABSTRACT

We provide the first replication study of political science research published in Brazil by attempting to replicate every quantitative article published in five major Brazilian journals between 2012 and 2016. We also tested whether replication rates varied between established fields, more traditional and where the use of quantitative data is more common, and emerging fields. Our results show that transparency and reproduction are still in a development stage in Brazilian Political Science. Of the 650 articles reviewed, we asked for data to 197 quantitative articles. From those, only 28% agreed to share datasets and computed codes. We were able to attempt a replication for only 14%, and successfully reproduce the results of less than 5%. We conclude by suggesting the adoption of transparency and replicability procedures that are standard in other scientific communities.

replication; transparency; reproduction; Brazilian Political Science; data sharing

RESUMO

Este é o primeiro exercício de replicação de pesquisas em ciência política publicado no Brasil. O objetivo foi replicar todos os artigos quantitativos publicados em cinco grandes revistas brasileiras entre 2012 e 2016. Também foram testadas as diferenças potenciais nas taxas de replicação entre campos de pesquisa estabelecidos, mais tradicionais e onde o uso de dados quantitativos é mais frequente, e emergentes. Solicitamos os dados para 197 artigos quantitativos. Desses 197, somente 28% dos autores concordaram em compartilhar os dados e os códigos. Foi possível tentar replicar apenas 14%, com 5% de sucesso. A conclusão sugere a adoção de procedimentos de transparência e replicabilidade já introduzidos em outras comunidades científicas.

replicação; transparência; reprodução; Ciência Política Brasileira; compartilhamento de dados

RESUMEN

Este es el primer ejercicio de replicación de la investigación en ciencia política publicado en Brasil. El objetivo fue replicar todos los artículos cuantitativos publicados en cinco importantes revistas brasileñas entre 2012 y 2016. También se probaron las posibles diferencias en las tasas de replicación entre los campos de investigación establecidos, más tradicionales y donde el uso de datos cuantitativos es más frecuente, y los emergentes. Solicitamos datos para 197 artículos cuantitativos. De estos 197, solo el 28% de los autores acordó compartir datos y códigos. Fue posible intentar replicar solo el 14%, con un 5% de éxito. La conclusión sugiere la adopción de procedimientos de transparencia y replicabilidad ya introducidos en otras comunidades científicas.

replicación; transparencia; reproducción; Ciencias Políticas Brasileñas; compartir datos

RÉSUMÉ

Il s’agit du premier exercice de réplication de la recherche en science politique publiée au Brésil. L’objectif était de reproduire tous les articles quantitatifs publiés dans cinq grandes revues brésiliennes entre 2012 et 2016. Les différences potentielles de taux de réplication entre les domaines de recherche établis et plus traditionnels et où l’utilisation de données quantitatives est plus fréquente et les domaines émergents ont également été testés. Nous avons demandé des données pour 197 articles quantitatifs. Sur ces 197, seuls 28% des auteurs ont accepté de partager des données et des codes. Il était possible d’essayer de reproduire seulement 14%, avec 5% de succès. La conclusion suggère l’adoption de procédures de transparence et de reproductibilité déjà introduites dans d’autres communautés scientifiques.

réplication; transparence; la reproduction; Science politique brésilienne; partage de données

INTRODUCTION

Scientific disciplines are facing a credibility crisis. In social sciences, significant published findings often cannot be replicated (Goodman, Fanelli and Ioannidis, 2016; Key, 2016; Christensen et al., 2019). High-profile publications have been retracted and, in some cases, identified as fraudulent. Some scholars have speculated that at least half of all published research is simply wrong (Ionnadis, 2005). The reproduction of other scholars’ work has led to several high-profile reviews and at least one significant retraction of a fraudulent result¹. All these (and other) problems suggest that much research is a waste of resources and human capital, degrade trust in science, and have led some to question public funding for research projects (Christensen and Miguel, 2018).

Yet, as Figueiredo Filho et al. (2019) pointed out, there is no consensus in the literature about the meaning of transparency, reproducibility, and replication, which leads these concepts to be treated as interchangeable. By and large, one assesses transparency by the research material available to the academic community. Reproducibility is determined by the capacity of any scholar to reproduce the same results from the research material made available. Finally, replication may have two different meanings. The first, used in this article, is a synonym of reproducibility. The second, according to King (1995), is the attempt to further knowledge by introducing slight variations in a reproducible paper, e.g., by adding variables to the original model, testing the same hypothesis in different contexts, etc.

In response, scholars have investigated the causes of this credibility crisis and suggested several solutions. The causes include institutional mechanisms that encourage “fishing” or p-hacking² (Humphreys, De La Sierra and Van Der Windt, 2013) and lead to selection bias in publication choice (Gerber, Green and Nickerson, 2001). For example, journals are more likely to publish “significant” findings than null results (Ferguson and Heene, 2012). Knowing this, scholars fail to submit null results for publication. They may also hunt through the dataset for significant results. All these mechanisms push relevant null results aside, encourage poor scientific practices, and lead to a high rate of false positives³ in published studies.

Addressing all these problems will require significant changes in scientific institutions. Scholars have proposed several reforms and best practices that have the potential to deal with these issues. These practices might include open access to academic production, data and research materials, computer codes (preferably in open source formats), open peer review, use of pre-registration and data analysis plans, reviewing research designs instead of results (Cruwell et al., 2019; Martins, 2020), and others promoted by organizations and initiatives, such as the Iniciativa Brasileira de Reprodutibilidade (https://www.reprodutibilidade.bio.br/), the Project TIER (https://www.projecttier.org/), the Berkeley Institute for Transparency in the Social Sciences (https://www.bitss.org/) and the Open Science Foundation (https://osf.io/).

Research transparency is one of these new practices. By research transparency we mean providing all the resources needed for other scholars to replicate one’s findings, including source materials, computer code, and other relevant resources (King, Keohane and Verba, 1994; Figueiredo Filho et al., 2019; Bakken, 2019). Transparency permits studies’ replication and reproducibility, which are key factors for research’s credibility (Janz, 2016). In the same way that transparency is a condition for research reproduction, reproducibility is the ability to reproduce the same results of a study using the same data and methodology (NSF, 2015)⁴.

Adopting transparency and reproducibility procedures would diminish the probability of mistakes (Simonsohn, 2013; King, 1995), increase both the citations rate (Piwowar, Day and Fridsma, 2007) and the trustfulness, efficiency and cumulative advance of academic work (King, 1995; King, 2006; Ball and Medeiros 2012, Finifter 1975; Goodman et al., 2015; Ebersol et al., 2016; Markowetz, 2015; Elman, Kapiszewski and Lupia, 2018; Christensen and Miguel, 2018; Gleditsch, 2020). The transparency and reproducibility culture are starting to emerge in some important fields, such as business and management sciences (Martins, 2020), psychological sciences (Gilbert et al., 2016), biomedical sciences (Bakken, 2019) and social sciences (Christensen, Freese and Miguel, 2019), as well as in great part of developed countries’ scientific communities (Degterev, 2020). Some journals already require the release of all replication materials, including data and computer code, as a condition of publication⁵. These requirements have significantly increased the replicability of research in those countries. However, little is known about replicability outside the United States and Western Europe.

Even with all this evidence showing that transparency helps to ensure academic research trustfulness, scientists are, in general, still suspicious about sharing their data with the community (Janz and Freese, 2019) or face many difficulties to guarantee free and usable access to them. The lack of sufficient resources (including technological frames and abilities, but also time and money) to make research data public available is one of the most cited reasons for scientists not to share their data (Tenopir et al., 2011). Besides, there is an important concern about the mistaken use of the shared data, specially as far as plagiarism is concerned (Huang et al., 2012). These reasons help us to understand why the rate of data sharing continues low, evidenced by empirical studies in different scientific fields, like less than 6% in behavioral sciences (Hildebrandt and Prenoveau, 2020), 10-13% in medical sciences (Savage and Vickers, 2009; Thelwall et al., 2020), 14% in psychological sciences (Hardwicke et al., 2020.), 36% in ecology and evolutionary sciences (Roche et al., 2015) and 40% in biological sciences (HUANG et al., 2012).

Using this conceptual framework, we report herein on the first research transparency and reproducibility analysis of political science in Brazil. We attempted to reproduce results of 197 articles reporting quantitative findings published between 2012 e 2016 in the five leading Brazilian political science and general social science journals, the Brazilian Political Science Review, the Revista de Ciência Política, Dados – Revista de Ciências Sociais, Opinião Pública, and Revista Brasileira de Ciências Sociais. Besides examining overall replication rates, we also observed differences in replicability across established, where the use of quantitative data is more common, and emerging subfields, as described below, as well as differences in the availability of data from publications. We tested for differences in methodological approach, transparency, and replicability between research in more and less established subfields in Brazilian Political Science.

Our results show that transparency and reproduction are still in a development stage in Brazilian Political Science. Of the 650 articles reviewed, we asked for data to 197 quantitative articles. From those, only 28% agreed to share datasets and computed codes. We were able to attempt a replication for only 14%, and successfully reproduce the results of less than 5%. Established subfield papers had higher transparency and reproduction rates than those for emerging fields, partly because they were more likely to use quantitative methods. For such articles, we found significantly higher rates of response, data provision, and replication success than with quantitative papers from emerging fields.

This study is divided into four sessions including this introduction. The next session presents the research design that includes the explanation of our hypotheses, criteria for classification of papers into fields (established or emerging) and methodology (quantitative or non-quantitative), details about the request for replication data and coding methodology for our dependent variables. The following session gives details about the replication procedure and the most important results this research has found. The last session is a conclusion of the study and it also brings some suggestions to improve transparency and replicability into the Brazilian social sciences community.

RESEARCH DESIGN

Conducting a replication study is not as trivial as some might think. Replications are difficult to conduct, time-consuming, and hard to publish (Janz, 2016). It is important to follow strict procedures to ensure that its design does not cause a failure in replicating. It seems to be the case of an important replication study conducted and reported by the Open Science Collaboration that attempted to replicate 100 published studies from psychological sciences fields and reported a surprisingly low reproducibility rate⁶. Gilbert et al. (2016) reported that this study’s design had errors in sampling, power and biases that would harm its conclusions.

To avoid this kind of error, we used a simpler, but accurate procedure. This study used the same dataset as used in original studies and tried to replicate the reported results without any change in samples or methodologies. We analyzed every paper published in the leading Brazilian Political Science journals, and coded each one according to its methodological approach, response to the data request, and research replicability⁷. We tried to replicate every paper that used a quantitative approach. The focus on “quantitative studies” is simply because such designs should make replication easier. When datasets and computer codes are archived appropriately, other analysts should be able to reproduce every figure or table very quickly. Rich and detailed historical studies may also be replicated, but the effort required to do so for five years of articles was beyond the capacity of our project.

Hypotheses

Our research was intended primarily as a measurement exercise to establish the extent of replication norms in Brazil. However, we also hypothesized that research in established subfields would be more replicable than research in emerging fields, reflecting the same biases discussed above. In established fields, null results are more likely to be published since they may challenge established scholarship. Similarly, the existence of a larger number of faculty focusing on core questions may increase scholars’ care in testing hypotheses, knowing others may question their work. In contrast, in emerging fields, manuscripts are less likely to be published unless they have significant results. Consider a study on soccer games and votes for incumbents. If there is a null result, the paper is unlikely to be published and may be dismissed as a bad idea. If there is a significant result, a journal may be inclined to publish a “clever” piece that may get media attention. In this way, emerging fields are more likely to suffer false-positive bias due to the institutional problems discussed above⁸.

We examined all 650 articles published between 2012 and 2016 in the five leading Brazilian political science and general social science journals: Brazilian Political Science Review, Revista Brasileira de Ciência Política, DADOS, Opinião Pública, and Revista Brasileira de Ciências Sociais. Figures 1 and 2 show the distribution of papers by journal and year, respectively.

Figure 1
Papers by Journal

Figure 2
Papers by Year

Identifying Articlefrom Established Subfields

To explore our hypotheses regarding mature and emerging areas of study, we first defined four research subfields: Electoral Politics, Legislative Politics, Political Parties, and Emerging/Other. The first three are among the most established not only in Brazilian Political Science, but also in the international academic community. The other categories included all emerging subfields. Then, we collected the keywords from all papers, and we selected the ones related to the established fields. We pre-determined categories based on the paper keywords, as ^{Appendix A} shows. From the 180 keywords, 88 (49%) were assigned to Electoral Politics, 58 (32%) to Legislative Politics, and 34 (19%) to Political Parties. In addition, we preregistered these categories at the Open Science Foundation. For example, we categorize keywords like “candidates”, “candidacies” and “election” as Electoral Politics; keywords like “House of Representatives”, “coalitions” and “Congress” as Legislative Politics and “party system”, “party funding” and “partisanship” were included in the Party Politics category.

We classified each paper into one or more of these four categories. From all 650 papers, 154 (24%) used one or more of the selected keywords. Among those, 60 (9%), 37 (6%), and 16 (2%) included keywords from one of the subgroups Electoral Politics (E), Legislative Politics keywords (L), and Political Parties (P) keywords, respectively. An additional 15 (2%) had keywords from both Electoral Politics and Legislative Politics (E/L), 21 (3%) papers included both Electoral Politics and Political Parties keywords (E/P), and two had both Legislative Politics and Political Parties keywords (L/P). Finally, three papers had all three keywords (E/L/P). The pieces that did not have any of the selected keywords were classified in the “Emerging” area of study.

Established subfields represented 23.7% of all 650 papers. The sum of each subfield is not equal to this percentage because a paper can be classified into more than one subfield. Note that the Electoral Studies subfield is slightly more popular than the other two established subfields. ^{Appendix B} has a detailed classification of each paper.

Figure 3
Papers by Subfields and Maturity

Identifying Quantitative Works

To implement our research design, we first needed to identify which papers had a quantitative approach that allowed replication using only dataset and computer codes. Our methodological decision to only try to replicate quantitative studies relied on the research’s limitations to both analyze and reproduce qualitative data⁹.

We sought the main results reported for each paper and evaluated if it included any quantitative analysis. We created a dummy variable named Quantitative and assumed value 1 if, after a basic overview of the study, we identified quantitative results among the study’s main findings. The presence of charts and tables would indicate the use of a quantitative methodology. This selection cannot be considered very strict for a couple of reasons. First, we did the work manually, since there was no way to collect the articles’ metadata automatically¹⁰. Second, in order to enlarge the number of observations, we decided to include studies that had relevant quantitative results but did not necessarily use a statistical inference analytical method. Therefore, we considered a study with only a descriptive analysis as quantitative.

This decision brought up a discussion about differences within the group of quantitative studies. It would be more challenging to replicate research that used complex econometric models than another that did only a descriptive analysis. We are aware of this situation, but, for this research, we chose not to explore this heterogeneity. It would be necessary to develop an objective criterion to classify each quantitative methodology by their complexity and separate them into groups according to different replication difficulties. These procedures were beyond our time and work resources.

Even with a soft parameter for Quantitative, the number of quantitative papers was only 197, or 30% of all pieces examined. Notably, the proportion of quantitative articles in established subfields was significantly higher than in emerging subfields. Although they were less than a quarter of the potentially replicable papers, the established subfield papers included more than half of the quantitative research.

Table 1 Quantitative papers by Maturity

	N	%	Total
Established	100	64.9%	154
Emerging	97	19.5%	496
Total	197	30.3%	650

Source: Research data. Elaborated by the authors.

Requesting Replication Materials

For each paper reporting quantitative results, we sent an e-mail message to the authors and requested a replication dataset and a computer code. The message explained the research goals and assured author privacy in disclosure of our research, using the model e-mail below:

“Subject: Request for Data for Replication Study Without Identification.

Dear Mr(s). (Author’s name),

We write on behalf of Professors George Avelino (FGV, coordinator of the Center of Politics and Economics of the Public Sector), and Scott Desposato (the University of California, San Diego, director of the Center for Latin American Studies at the same university). We are developing an analysis of the replicability of the Political Science articles published in the leading Brazilian journals in the field, and your paper is in our database.

We would like to request both the dataset and the computer codes required to reproduce the results in your article (Title of the article), published in the journal (Name of the journal) on year (Year), volume (Volume number), (Journal number).

Your name, article name, or any other identifying information about you or your article, will NOT be disclosed in our final report.

Thank you in advance for your contribution to our project. Regards,”

The procedure followed the schedule below:

Phase I (responses)

· March 5^th, 2018: sent the original message;

· March 26^th, 2018: sent the first follow-up message to those who did not answer the original message and to address any questions;

· April 20^th, 2018: sent a second follow-up message to those who did answer the first follow-up message and to answer any remaining questions;

· April 27^th, 2018: deadline for receiving answers.

Phase II (completeness)

· May 7^th, 2018: sent a second message asking for the complete material for those who answered our first e-mail, but who provided incomplete replication materials;

· May 21^st, 2018: final deadline for answers for incomplete material.

In total, scholars had almost eight weeks answer our first message, and 11 weeks to provide replication materials. In general, the ones who actually answered our request did it very shortly. A more extended period might have increased our answer rate, but we received no additional answers after the final deadline.

Coding

Besides the proportion of studies that used a quantitative approach, we examined four transparency and replicability measures applied to each article that we characterized as having some quantitative analysis. Our four measures were:

· Responsiveness: Did the author(s) answer our email, even with a refusal to collaborate?

· Agreement: Did the author(s) agree to share data?

· Completeness: Did the author(s) provide enough data and code to enable the research replication attempt?

· Replicability: Were results replicable?

For Responsiveness, we first coded the papers by combining the answers in the two research phases (the first message and the follow-up) and the lack thereof: 1) Answered in phase 1; 2) Answered in phase 2; 3) Did not answer. Then, we coded Responsiveness as “1” for those that answered our request either in Phase 1 or Phase 2. In total, 83 of the 197 papers (42%) answered our request. They were divided nearly equally between established and emerging fields.

The second variable, Agreement, measures whether the author agreed to share the data or not. The variable had six categories: 1) Answered with data; 2) No response; 3) Data is not available; 4) Cannot share data; 5) Refusal and 6) Other. We coded the variable Agreement as a “1” for articles where the author provided data. This variable was not among our pre-registered dependent variables, but it seemed appropriate to distinguish the authors who answered positively from those who did not. Among the 83 responses, 56 (67.5%) provided some analysis data. From these, 36 papers (almost 65% of the 56) were from established fields.

As far as Completeness was concerned, we coded the following categories: 1) Complete; 2) No useful data; 3) Only code; 4) Only data source, and 5) Only dataset. We coded Completeness as a “1” for articles in which authors sent at least one dataset and some computer code. Only half of the 56 papers that provided data did it completely. More than two-thirds of these papers came from established fields (19).

Finally, Replicability had three categories: 1) Completely replicable (all results were reproduced, including sign and statistical significance); 2) Partially replicable (not all results were completely reproduced, or signs/significance did not match the ones reported in the paper) or 3) Non-Replicable (no result could be reproduced). We coded Replicability as “1” for articles in which data that the author(s) provided produced the same results in terms of values, signs, and statistical significance. From those 28 papers with complete data, we were able to fully replicate only 10 (35.7%), seven of which were from established fields.

Table 2 presents how dependent variables were coded according to the situations described above. Only studies that were coded “1” for Replicability were considered successfully replicable. Table 3 brings descriptive variables by field type for our sample (quantitative studies) and dependent variables.

Table 2
Dependent Variables Coding

Situation Description	Responsiveness	Agreement	Completeness	Replicability
Author(s) did not answer data request	0	0	0	0
Author(s) answered data request but did not agree to share data	1	0	0	0
Author(s) answered data request and shared data, but data was not complete (dataset or codes missing)	1	1	0	0
Author(s) answered data request and shared complete data but not all reported results were replicable at same statistical significance, value, and sign.	1	1	1	0
Author(s) answered data request and shared complete data but and all reported results were replicable at same statistical significance, value, and sign.	1	1	1	1

Table 3
Descriptive Variables

	Emerging	Established	Total
All Articles	496	154	650
Quantitative	97 (20%)	100 (65%)	197 (30%)
Responsiveness	34 (7%)	49 (32%)	83 (13%)
Agreement	20 (4%)	36 (23%)	56 (9%)
Completeness	9 (2%)	19 (12%)	28 (4%)
Replicability	3 (1%)	7 (5%)	10 (2%)

Source: Research data. Elaborated by the authors.

Replication Procedure

We attempted to replicate all papers for which authors shared data and, in some cases, code. The first decision was to define which quantitative results to replicate, since many papers had multiple charts and tables with data not related to the core results. Given the magnitude of the task, we chose to focus on the paper’s prominent results. To identify a prominent result, we identified quantitative findings that were most discussed, especially in the abstract, introduction, and conclusion of the study. This procedure constrained our analysis to a smaller number of charts and tables. It also meant we ignored difficulties in replicating secondary analyses.

In replicating, we found problems with data, codes, or results. Figure 4 presents the distribution of these problems. Of the 28 for which we could attempt a replication, we found problems with almost two-thirds of them (18). The most frequent had to do with computer codes, which represented about one-third (10). We also had problems with results and data in eight (28.6%) and seven papers (25%), respectively. Finally, seven articles (21.5%) presented more than one type of problem.

Figure 4
Papers Replicated by Replication Problems

The types of coding problems involved either a non-working or an incomplete statistical computing code. In some cases, we could not calculate a specific result displayed in the paper. In others, there were syntax errors that made a complete replication impossible. We frequently received code lacking one or more calculations, and in many cases, poorly documented, with analyses and data management left unexplained. In one case, the data analysis was computationally intensive and eventually produced results like those published, but it did not reproduce the central figures. Interestingly, many code sets led to results that were not part of the article, suggesting some fishing for results. Finally, there were some minor syntax errors that we were able to correct.

Data problems involved cases in which materials that the authors shared lacked a critical variable or even a key dataset. For example, one author provided a dataset which seemed to be a complete dataset (data frame and code) and which could replicate some of the published regressions. However, further results in the computer code required an additional dataset, which the authors did not share. In some cases, the authors shared a web address where we could find the raw data, but they did not provide specific files with instructions for its coding and cleaning. In other cases, the authors provided a complex code with no instructions on how to execute it.

Some cases were more problematic since we could not open the dataset, or the data differed from the published results. In other cases, authors shared many different files, but none was useful or even required by the accompanying computer code. One author shared a zip file with multiple “xls” files from which we could not replicate results. These papers were classified as “no useful data” and did not proceed to the replication phase.

In some other cases, although we were able to produce results, they did not match those published. We classified those cases as results problems. Some codes failed to define the data subset used for analysis, and the number of observations displayed in the tables did not match with our results. In such cases, even descriptive statistics could not be reproduced. Other papers had problems with the statistical results. For example, our reproduction of regression models yielded different coefficient values for one or more variables.

Data was basically provided in Excel, Stata and R formats, which are well known statistical software. Except for minor corrections in scripts (including adaptations to software updates), replicators were oriented to not modify the code structure. Dataset and code organization is an important part regarding data transparency. In the case an author shared files in a different format or in a way that was not possible to understand how to use it, they were asked to provide more information. In cases that it happened, authors did not answer for our help request and the data were considered as “non-useful”.

Successful replications had several shared features. Most notably, the authors provided a well-organized set of data and codes. In one case, the authors did not send the computer code, but their results were clearly described in the text of their article and were straightforward to reproduce. In another case, the authors provided the dataset in a “xls” format file and a Stata file with few lines. Those lines described what table or graph it was related to in the paper. This type of information was critical to enable replication.

Established versus Emerging Quantitative Subfields

Of the 650 papers examined, 197 (30.3%) used a quantitative approach. Of these, only 56 (28.4% of quantitative papers and 8.6% overall) responded positively to our request for data. In half of these cases, the data was incomplete, which did not allow replication. Most of these were cases in which the author shared only codes or only a dataset, but not both. In the end, we completed a replication attempt on 28 papers (14.2% of quantitative papers and 4.3% overall), and only ten of them were fully replicable. In those last cases, the authors provided all data and code to reproduce the core results in the paper. In other words, we were able to successfully replicate only 5% of quantitative and 1.5% of all papers.

We hypothesized that established quantitative subfields would have stronger replication norms and higher rates of replicability than emerging fields. We tested our hypothesis by comparing our measures of responsiveness, agreement, data provision, and reproducibility for both established and emerging fields. Table 4 results from T-tests for a difference of means between these two groups among the five dependent variables.

Table 4
T-Tests on Dependent Variables by Maturity (Emerging – Established)

	Quantitative	Responsiveness	Agreement	Completeness	Replicability
Coef.	-0.454*	-0.234*	-0.178*	-0.105*	-0.039*
S.E	(0.039)	(0.029)	(0.024)	(0.018)	(0.011)
N	650	650	650	650	650

• * |P|>5%
Source: Research data. Elaborated by the authors.

For all our dependent variables, the proportions are higher for the established than for the emerging fields. Authors of research in established areas were nearly twice as likely to respond to our request with some data (36% versus 20.6% in emerging fields). Additionally, 52.8% of authors from those fields provided complete data, versus 45% of those in emerging ones. Finally, of those that provided data, over half of the results from established fields could be replicated (52.8%), versus a slightly lower rate for emerging areas (45%).

The difference of means test presented in Table 3 showed that established subfields papers have a significantly higher probability of adopting a quantitative methodology, answering a request for data, providing the data with more quality, and ultimately being replicable. When including control variables in an OLS regression, the results are similar.

Table 5 reports the results of OLS regressions of our five dependent variables. Independent variable of interest is Established, a dummy variable that indicates papers from established subfields. Control variables include the number of authors and dummies for journals and year of publication.

Table 5
OLS Regressions Results

	Quantitative	Responsiveness	Agreement	Completeness	Replicability
Established	0.366*	0.183*	0.142*	0.091*	0.030*
	(0.037)	(0.029)	(0.024)	(0.019)	(0.012)
Authors	0.101*	0.031*	0.021	-0.013	-0.006
	(0.019)	(0.015)	(0.013)	(0.010)	(0.006)
J_OP	0.071	0.128*	0.050	0.009	0.030
	(0.058)	(0.045)	(0.038)	(0.029)	(0.018)
J_D	-0.063	-0.019	-0.059	-0.051	-0.024
	(0.054)	(0.042)	(0.042)	(0.028)	(0.017)
J_RBCP	-0.273*	-0.080	-0.084*	-0.077*	-0.025
	(0.055)	(0.043)	(0.036)	(0.028)	(0.017)
J_RBCS	-0.191*	-0.056	-0.076*	-0.061*	-0.015
	(0.055)	(0.043)	(0.036)	(0.028)	(0.017)
Y_2013	-0.049	-0.043	-0.015	0.013	0.011
	(0.049)	(0.038)	(0.032)	(0.025)	(0.015)
Y_2014	-0.007	0.034	0.050	0.048	0.032*
	(0.049)	(0.038)	(0.032)	(0.025)	(0.015)
Y_2015	0.055	0.017	0.058	0.061*	0.023
	(0.049)	(0.038)	(0.032)	(0.025)	(0.015)
Y_2016	0.020	0.041	0.063*	0.060*	0.016
	(0.048)	(0.038)	(0.032)	(0.024)	(0.015)
Constant	0.164*	0.031	0.021	0.047	0.011
	(0.064)	(0.050)	(0.042)	(0.033)	(0.020)
N	650	650	650	650	650
F Test	27,83	12,39	10,2	6,19	3,43
R-squared	0.30	0.16	0.14	0.09	0.05

• * |P| > 5%
Source: Research data. Elaborated by the authors.

Established variable remains positive and significant for all dependent variables. The control variables do not have any clear patterns¹¹. The number of authors had a positive and significant effect only on whether articles included quantitative analysis and whether any of them answered our e-mail. Using Brazilian Political Science Review as baseline, the reported effect of journals shows that both Revista Brasileira de Ciência Política and Revista Brasileira de Ciências Sociais seem to have a lower rate of quantitative papers.

Additionally, authors from the same journals were generally less likely to agree to share their data and to share complete data. Finally, results for year of publication have 2012 as baseline and do not strongly suggest an increasing replicability in more recent papers. Only Agreement (2015 and 2016) and Completeness (2016) reported significant higher averages for more recent papers.

CONCLUSION

Around the world, the credibility crisis in science poses a challenge to scholars’ efforts to advance knowledge both in the natural and the social disciplines. As Figueiredo et al. (2019) discussed, norms of transparency and replicability are becoming part of standard scientific practice in the United States, Western Europe, and other regions. The scientific community cannot neglect credibility issues anymore. Addressing this crisis will require significant changes in the procedures of scientific institutions, journals, and scholars. There are reforms and best practices that should spread to all academic communities. Especially noteworthy is a general norm of research transparency.

This study is the first look at transparency and replicability in Brazil. It covers only a small sample of the academic work produced in the country and it has its limitations in terms of generalization and comparison with other countries’ standards. Even so, the evidence presented here can trigger new research in this field that would help the Brazilian academic community to follow this critical path.

Let us make it clear: our findings do not mean most research is invalid, or conclusions are incorrect. It is just a check on how easy it is to replicate or reproduce the main results. Strictly, if one considers the time lag between submission and publication, our data may even not represent the field accurately during the period¹². Besides, we only check whether scholars provide the data we requested within two months of our request and whether we can easily use it to generate results identical to those in publication. With more time and more explanation of our purposes, perhaps we would have a higher response rate. With some sources from which we did receive data but failed to replicate results, with more time, maybe we could have reverse engineered findings and asked for more assistance from authors in understanding what they provided. With more time, we could have attempted to replicate results by collecting the original data ourselves from their sources. Again, non-reproducible does not mean invalid in any way. We believe it primarily reflects a lack of transparency norms in Brazilian political science and the need for a new standard according to which it should be easy to replicate and verify any published findings.

The difference in the results from established and emerging fields was also somewhat expected. In general, the more scholars engage in a dialogue around common themes and questions, and the more the field often uses quantitative analysis, the higher their need for transparency procedures to enhance the collective advancement of knowledge.

Admittedly, introducing replication practices in qualitative research poses a different challenge, which would go beyond the scope of this paper. Despite the methodological differences involved, this would be a great improvement for the Brazilian political science community, due to its large number of qualitative works. Also, introducing transparency practices for qualitative research would help to keep pace with the increasing use of mixed methods research designs, which usually requires teams made with members from different methodological backgrounds. Within this picture, trying to define transparency and replication practices in qualitative research, without compromising its small “n” and interpretive characteristics, is a crucial step. (Sukumar and Metoyer, 2019; Moravcsik, 2014a, 2014b, 2019).

Our replicability analysis in Brazil suggests that political scientists in that country need to adopt transparency practices. Finally, we offer several recommendations for scholars and journals to improve replicability and maturity in these fields.

Authors can publish a Pre-Analysis Plan or a Registered Report before the beginning of their data collection or analysis (Casey, Glennerster and Miguel, 2012; Miguel et al., 2014), even on qualitative studies (Pineiro and Rosenblatt, 2016). This procedure will strengthen research transparency, mitigating some of the credibility problems we found. Explicitly specifying the research question and defining our analysis plan will make researchers pay more attention to their study design and focus on theory and research questions. These procedures are relatively easy to adopt, since there are platforms like the Open Science Framework, where one can pre-register research plans and share data, codes, and results.

Scientific institutions can teach and encourage their members to implement these procedures (Steinhardt, 2020). Undergraduate, Graduate and Ph.D programs should have replication classes and should teach the best practices for data management¹³. These classes would increase the students’ chances to get their research proposals funded, since a growing number of funding institutions began to require the inclusion of a data management plan in the submitted proposals. For instance, FAPESP, a well-known Brazilian research funding institution, began to require the inclusion of a data management plan (http://www.fapesp.br/gestaodedados/). Additionally, data management courses could also provide an easy way for students to get their first publication by replicating other papers (King, 2006). Finally, a recent paper by Christensen et al. (2019) estimates that papers that make data and coding available receive additional scholarly citations.

Journals can support the publication of null results and of review processes that focus on research questions and design, rather than provocative results. For example, the Center for Open Science recommends that journals include peer review of research plans as well as results, known as Registered Reports:

Registered Reports is a publishing format used by over 200 journals that emphasizes the importance of the research question and the quality of methodology by conducting peer-review before data collection. High-quality protocols are then provisionally accepted for publication if the authors follow through with the registered methods. This format rewards best practices in adhering to the hypothetical-deductive model of the scientific method. It eliminates a variety of questionable research practices, including low statistical power, selective reporting of results, and publication bias, while allowing complete flexibility to report serendipitous findings. (https://cos.io/rr/)

Registering a research plan is not an instrument to restrict scientists on their flexibility to conduct their research or a safeguard for journals against their responsibility for the credibility of their publications. Unpredictable events may occur during data collection, imposing changes in analysis strategy. A pre-registered report informs readers about such unforeseen problems faced by researchers, and solutions implemented, enhancing research transparency (Christensen, Freese and Miguel, 2019). In other scenarios, readers never learn the real reason behind the methodological choice, opening the way for credibility questioning. For journals, pre-registered analysis plans are an opportunity to mitigate publication bias and reduce incentives for fishing and p-hacking.

Officially, releasing replication data is another essential practice to improve science transparency. Using and sharing computer codes allow research verification and replication (King, 2003; Elman, Kapiszewski and Lupia, 2018; Dafoe, 2014). Publishing replication materials increases incentives for careful analysis and accurate reporting. Verification also allows for investigating potential cases of fraud. Indeed, it would have been impossible to uncover recent cases of data fabrication in the United States scandals without access to underlying data. Replication materials can also support academic teaching and learning.

To keep in touch with the international trend, the Brazilian scientific community should incorporate transparency practices. ABCP (Associação Brasileira de Ciência Política), as well as other associations and Journals, should adopt standards of replication, requiring and archiving data and replication code for published research, subject to the privacy and copyright restrictions.

They could endorse or even implement together a general data repository to concentrate all scientific work. A well-known benchmark is The Berkeley Initiative for Transparency in the Social Sciences (BITSS).

The Berkeley Initiative for Transparency in the Social Sciences (BITSS) is another new institution that has emerged in recent years to promote dialogue and build consensus around transparency practices. BITSS has established an active training program for the next generation of economists and other social scientists, as well as an award to recognize emerging leaders in this area, the Leamer-Rosenthal Prize for Open Social Science (Christensen and Miguel, 2018:61-62).

Adopting these recommendations will increase the maturity and credibility of social science in Brazil and benefit scholarship and society. Associations and journals can both encourage and facilitate such practices. For example, ABCP and universities can offer courses to teach their members how to track their research procedures, how to prepare a dataset, and how to use computer programs that facilitate this task. In addition, to further transparency, ABCP could lead an association with an internationally known data storage, such as Dataverse (https://dataverse.org/), which could increase the rewards for registering dataset and codes by making them available to the international scientific community. Universities need to nurture a reproducibility and replication culture within their departments and courses to ensure that a “gold standard” of credibility in academia is strengthened (Janz, 2016). They can also provide online storage for datasets and code sharing.

To further the publication of datasets that interest a broad audience, journals’ editors can create a section dedicated to papers presenting datasets to the community. Journals can also publicly require datasets and codes to submission approval (Dafoe, 2014). Finally, journals need to encourage replication publications, but not implement a reverse publication bias, focusing only on failed replications attempts (Janz and Freese, 2019). Recently, the Brazilian Political Science Review adopted a policy that supported transparency. From their author’s guideline for submission:

Authors submitting empirical papers should provide, when and if the manuscript is approved, the database used in the analysis, the code dictionary describing the variables, and the code for replication or the series of steps of the analysis. The database must be in an easily accessible format for conventional statistical software – such as R, Stata, SPSS, and Excel. The code dictionary should indicate the name of the variable, its description, and the source of the data. Where appropriate, the code for replication of the analyzes should be extensively commented. When the article does not use statistical software that allows the creation of a replication code, the author should describe the procedures, step-by-step, for replication to be possible.

In a good note, a more recent survey with the editors of all other four journals investigated in this research shows they are all intending to introduce transparency and replicability procedures soon. This result shows that concerns with those procedures are starting to prevail in the Brazilian Political Science, making it closer to the international community. A future analysis will tell whether this is an excessive optimistic vision or not.

The scientific community is increasingly adopting transparency norms. Most likely, they will become mandatory for international publication and international research funding. The sooner Brazil’s scientific community incorporates these transparency standards, the better. In short, implement transparency procedures – like research pre-registration and sharing of codes and data – is beneficial for everyone. It mitigates credibility suspicions, values scientific work, and, above all, allows research activity to concentrate on what is most important: the questions they seek to answer.

REFERENCES

• BALL, Richard; MEDEIROS, Norm. (2012), “Teaching integrity in empirical research: a protocol for documenting data management and analysis”. The Journal of Economic Education, vol. 43, n^o 02, pp. 182-189.
• BAKKEN, Suzanne. (2019), “The journey to transparency, reproducibility, and replicability”. Journal of the American Medical Informatics Association, vol. 26, Issue 3, pp. 185-187.
• CASEY, Katherine; GLENNERSTER, Rachel; MIGUEL, Edward (2012), “Reshaping institutions: evidence on aid impacts using a pre-analysis plan”. The Quarterly Journal of Economics, vol. 127, n^o 4, pp. 1755-1812.
• CHRISTENSEN, Garret; MIGUEL, Edward (2018), “Transparency, reproducibility, and the credibility of economics research”. Journal of Economic Literature, vol. 56, n^o 3, pp. 920-980.
• CHRISTENSEN, Garret; FREESE, Jeremy; MIGUEL, Edward (2019), Transparent and reproducible social science research: how to do open science. Oakland, CA: University of California Press.
• CHRISTENSEN, Garret et al. (2019), “A Study of the Impact of Data Sharing on Article Citations Using Journal Policies as a Natural Experiment”. PLoS ONE, vol. 14, n^o 12, e0225883. https://doi.org/10.1371/journal.pone.0225883
  » https://doi.org/10.1371/journal.pone.0225883
• CHORDIA, Tarun; GOYAL, Amit; SARETTO, Alessio. (2017), “P-hacking: evidence from two million trading strategies”. Swiss Finance Institute Research Paper, n. 17-37.
• CRUWELL, Sophia et al. (2019), “Seven easy steps to open science: an annotated reading list”. Zeitschrift für Psychologie, vol. 227, n^o 4, pp. 237-248.
• DAFOE, Allan. (2014), “Science deserves better: the imperative to share complete replication files”. Political Science and Politics, vol. 47, n^o 1, pp. 60-66.
• DEGTEREV, Denis A. (2020), Replicability of Research on International Relations As a Global Trend. Herald of the Russian Academy of Sciences, vol. 90, pp. 36-44.
• ELMAN, Colin; KAPISZEWSKI, Diana. (2014), “Data Access and Research Transparency in the qualitative tradition”. PS: Political Science & Politics, vol. 47, n^o 01, pp. 43-47.
• FERGUSON, Christopher J.; HEENE, Moritz. (2012), “A vast graveyard of undead theories: publication bias and psychological science’s aversion to the null”. Perspectives on Psychological Science, vol. 07, n^o 06, pp. 555-561.
• FIGUEIREDO FILHO, Dalson et al. (2019), “Seven reasons why: a user’s guide to transparency and reproducibility”. Brazilian Political Science Review [Online], vol. 13, n^o 02. (15/12/2019), pp. 1-37. Available in: http://dx.doi.org/10.1590/1981-3821201900020001 ISSN 1981-3821.
  » http://dx.doi.org/10.1590/1981-3821201900020001
• FINIFTER, Bernard M. (1975), “Replication and extension of social research through secondary analysis”. Social Information, vol. 14, n^o 02, pp. 119-153.
• GERBER, Alan S.; GREEN, Donald P.; NICKERSON, David. (2001), “Testing for Publication Bias in Political Science”. Political Analysis, vol. 09, n^o 04, pp. 385-392.
• GILBERT, Daniel T. et al. (2016), “Comment on Estimating the Reproducibility of Psychological Science”. Science, vol. 351, n^o 6277, p. 1037.
• GLEDITSCH, Kristian S. (2020), Houston, we have a problem: enhancing academic freedom and transparency in publishing through post-publication debate”. Political Studies Review. DOI: 10.1177/1478929919889309.
• GOODMAN, Alissa et al. (2015), “Ten simple rules for the care and feeding of scientific data”. PLOS Computational Biology, vol. 10, n^o 04, pp. 01-05.
• GOODMAN, Steven N.; FANELLI, Danielle; IOANNIDIS, John P. A. (2016), “What does research reproducibility mean?”. Science Translational Medicine, vol. 08, n^o 341, pp. 341ps12.
• HAMERMESH, Daniel (2007), “Replication in Economics”. NBER Working Paper, n. 13026.
• HARDWICKE, Tom E. et al. (2020), Estimating the Prevalence of Transparency and Reproducibility-Related Research Practices in Psychology (2014-2017). Available in https://osf.io/preprints/metaarxiv/9sz2y/
  » https://osf.io/preprints/metaarxiv/9sz2y/
• HEAD, Megan L. et al. (2015), “The Extent and Consequences of P-Hacking in Science”. PLOS Biology 13(3): e1002106. https://doi.org/10.1371/journal.pbio.1002106
  » https://doi.org/10.1371/journal.pbio.1002106
• HILDEBRANDT, Tom; PRENOVEAU, Jason M. (2020), “Rigor and reproducibility for data analysis and design in the behavioral sciences”. Behavior Research and Therapy, vol. 126. doi:10.1016/j.brat.2020.103552
• HUANG, Xiaolei et al. (2012), “Willing or unwilling to share primary biodiversity data: results and implications of an international survey”. Conservation Letters, vol. 05, n^o 05, pp. 399-406.
• HUMPHREYS, Macartan; DE LA SIERRA, Raul Sanchez; VAN DER WINDT, Peter. (2013), “Fishing, commitment, and communication: a proposal for comprehensive nonbinding research registration”. Political Analysis, vol. 21, n^o 01, pp. 1-20.
• IONNADIS, John. (2005), “Why most published research findings are false”. PLOS Medicine. Available at: https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
  » https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
• JANZ, Nicole. (2016), “Bringing the gold standard into the classroom: replication in university teaching”. International Studies Perspectives, vol. 17, issue 4, pp. 392-407.
• JANZ, Nicole; FREESE, Jeremy. (2019), “Good and bad replications in political science: how replicators and original authors (should) talk to each other”. Available in https://www.mzes.uni-mannheim.de/openscience/wp-content/uploads/2019/01/Janz-Freese_-Good-and-Bad-Replications-1.pdf
  » https://www.mzes.uni-mannheim.de/openscience/wp-content/uploads/2019/01/Janz-Freese_-Good-and-Bad-Replications-1.pdf
• KEY, Ellen M. (2016), “How are we doing? data access and replication in political science”. PS: Political Science and Politics, vol. 49, n^o 02, pp. 268-272.
• KING, Gary. (1995), “Replication, Replication”. PS: Political Science and Politics, vol. 28, n^o 03, pp. 444-452.
• KING, Gary. (2003), “The Future of the Replication Movement”. International Studies Perspectives, vol. 04, n^o 1, pp. 72-107.
• KING, Gary. (2006), “Publication, Publication”. PS: Political Science and Politics, vol. 39, n^o 01, pp. 119-125.
• KING, Gary; KEOHANE, Robert O.; VERBA, Sidney. (1994), Designing social inquiry: scientific inference in qualitative research. Princeton: Princeton University Press.
• MARKOWETZ, Florian. (2015), “Five selfish reasons to work reproducibly”. Genome Biology, vol. 16, n^o 01, p. 274.
• MARTINS, Henrique Castro. (2020), “A Importância da Ciência Aberta (Open Science) na Pesquisa em Administração”. Rev. Adm. Contemp, vol. 24 n^o 01.
• MIGUEL, Edward et al. (2014), “Promoting Transparency in Social Science Research”. Science, vol. 343, n^o 166, pp. 30-31.
• MORAVCSIK, Andrew. (2014a), “Transparency: The Revolution in Qualitative Research”. PS: Political Science and Politics, vol. 47, n^o 1, pp. 48-53.
• MORAVCSIK, Andrew. (2014b), “Trust but verify: the transparency revolution and qualitative international relations”. Security Studies, vol. 23, n^o 4, pp. 663-688.
• MORAVCSIK, Andrew. (2019), Transparency in Qualitative Research. SAGE Research Methods (https://www.princeton.edu/~amoravcs/library/TransparencyinQualitativeResearch.pdf
  » https://www.princeton.edu/~amoravcs/library/TransparencyinQualitativeResearch.pdf
• NSF (National Science Foundation). (2015), Reproducibility, Replicability, and Generalization in the Social, Behavioral, and Economic Sciences. Available at <https://www.nsf.gov/sbe/SBE_Spring_2015_AC_Meeting_Presentations/Bollen_ Report_on_Replicability_SubcommitteeMay_2015.pdf>.
  » https://www.nsf.gov/sbe/SBE_Spring_2015_AC_Meeting_Presentations/Bollen_ Report_on_Replicability_SubcommitteeMay_2015.pdf
• PINEIRO, Rafael; ROSENBLATT, Fernando. (2016), “Pre-Analysis Plans for Qualitative Research”. Revista de Ciência Política (Santiago) [online], vol. 36, n^o 3, pp. 785-796.
• PIWOWAR, Heather A.; DAY, Roger S.; FRIDSMA, Douglas B. (2007), “Sharing Detailed Research Data Is Associated with Increased Citation Rate”. PloS One, vol. 2, n^o 3, p. e308.
• ROCHE, Dominique G. et al. (2015), “Public data archiving in ecology and evolution: how well are we doing?”. PLoS Biol, vol. 13, n^o 11, p. e1002295.
• SAVAGE, Caroline J.; VICKERS, Andrew J. (2009), “Empirical study of data sharing by authors publishing in Plos Journals”. PloS One, vol. 4, n^o 9, p. e7078.
• SIMONSOHN, Uri. (2013), “Just post it: the lesson from two cases of fabricated data detected by statistics alone”. Psychological Science, vol. 24, n^o 10, pp. 1875-1888.
• SUKUMAR, Porna Talkad; METOYER, Ronald. (2019), “Replication and transparency of qualitative research from a constructivist perspective”. OSF Preprints (https://osf.io/6efvp)
  » https://osf.io/6efvp
• TENOPIR, Carol et al. (2011), “Data sharing by scientists: practices and perceptions”. PloS One, vol. 6, n^o 6, p. e21101.
• THELWALL, Mike et al. (2020), “Is useful research data usually shared? an investigation of genome-wide association study summary statistics”. Plos One, vol. 15, n^o 2, p. e0229578.
• TUVAL-MASHIACH, Rivka. (2017), “Raising the curtain: the importance of transparency in qualitative research”. Qualitative Psychology, vol. 4, n^o 2, pp. 126-138.

NOTES

• 1
  . See Lacour and Green fraud case in https://www.chronicle.com/article/What-Social-Science-Can-Learn/230645
• 2
  . For more information about p-hacking concept see Head et al. (2015) and Chordia, Goyal and Saretto (2017).
• 3
  . A false positive result consists in rejecting the null hypothesis when it was not the case of rejecting it. The opposite case is known as false negative, when the null hypothesis is not rejected when it should be.
• 4
  . Though Hamermesh (2007) separates simple from scientific replication, designating the last term to the expansion of scientific knowledge, we are more concerned with the first term, meaning transparency and reproducibility. Therefore, throughout this paper, we use replication and reproducibility interchangeably.
• 5
  . More than 1,100 journals have implemented at least one of the Transparency and Openness Promotion Guidelines, according to the Center of Open Science website. Some of the journals are American Journal of Political Science, European Political Science, Italian Political Science Review, Quarterly Journal of Political Science, State Politics and Policy Quarterly, Public Administration Review, Political Science Research and Methods, Political Behavior, Law and Human Behavior, Journal of Economic Behavior & Organization, Political Science Research and Methods. The complete list can be accessed in https://www.cos.io/our-services/top-guidelines?
• 6
  . Research Articles, August 28th, 2015, aac4716.
• 7
  . Since we covered the five leading journals for five years, increasing either number of journals or years should not make a difference for our analysis.
• 8
  . This example is provided for illustration purposes only and is not intended to comment on any research project.
• 9
  . For more details on the challenges of implementing reproduction and replication of qualitative studies see Elman and Kapiszewski (2014), Tuval-Mashiach (2017) or https://www.qualtd.net/ and https://openworking.tudl.tudelft.nl/2019/02/11/what-does-reproducibility-mean-for-qualitative-research/
• 10
  . We thank an anonymous reviewer for pointing us that there is a R package that can extract metadata from journals today. See more details in https://fmeireles.com/blog/rstats/rscielo-um-scraper-para-extrair-dados-scielo.
• 11
  . We did not preregister hypotheses about control variables.
• 12
  . We thank an anonymous reviewer for calling our attention to this point.
• 13
  . See https://www.academia.edu/42204205/Transparency_and_Reproducibility_in_Empirical_Research_2020.1_ for a good example of course structure provided by Professor Dalson Britto Figueiredo Filho of Universidade Federal de Pernambuco.
• *
  Hypotheses and data analysis plan were pre-registered at the Open Science Foundation except as noted. Analysis of existing datasets was exempted by the UCSD Institutional Review Board (IRB). The authors thank the Berkeley Institute for Transparency in the Social Sciences for funding and support. George Avelino and Ivan Mardegan acknowledge the support from São Paulo Research Foundation FAPESP (Grants #: 2013/156158-1 and 2017/20855-1). The authors also thanks Raduan Meira for research assistance.

APPENDIX

A – Keywords Classification

Keyword	Field
2010 Brazilian elections	Electoral Politics
2010 elections	Electoral Politics
2014 Brazilian elections	Electoral Politics
2014 elections	Electoral Politics
Biracial electoral races	Electoral Politics
Brazilian elections	Electoral Politics
Brazil’s presidential election in 2014	Electoral Politics
Candidacies	Electoral Politics
Candidates	Electoral Politics
Candidates for deputy	Electoral Politics
Career	Electoral Politics
Career patterns	Electoral Politics
Coattail effect	Electoral Politics
Correct voting	Electoral Politics
Economic vote	Electoral Politics
Economic voting in Brazil	Electoral Politics
Election	Electoral Politics
Election campaigns	Electoral Politics
Election expenses	Electoral Politics
Election studies	Electoral Politics
Elections	Electoral Politics
Elections 2014	Electoral Politics
Electoral Arena	Electoral Politics
Electoral behavior	Electoral Politics
Electoral behavior	Electoral Politics
Electoral behavior in Brazil	Electoral Politics
Electoral campaign	Electoral Politics
Electoral campaigns	Electoral Politics
Electoral competition	Electoral Politics
Electoral coordination	Electoral Politics
Electoral costs	Electoral Politics
Electoral effects	Electoral Politics
Electoral financing	Electoral Politics
Electoral fraud	Electoral Politics
Electoral geography	Electoral Politics
Electoral governance	Electoral Politics
Electoral information	Electoral Politics
Electoral instability	Electoral Politics
Electoral justice	Electoral Politics
Electoral justice system	Electoral Politics
Electoral performance	Electoral Politics
Electoral realignment	Electoral Politics
Electoral recruitment	Electoral Politics
Electoral rules	Electoral Politics
Electoral strategies	Electoral Politics
Electoral Studies	Electoral Politics
Electoral success	Electoral Politics
Electoral systems	Electoral Politics
Electoral television slots	Electoral Politics
Electoral volatility	Electoral Politics
Evangelical vote	Electoral Politics
Female candidacies	Electoral Politics
Geographical scattering of votes	Electoral Politics
HGPE	Electoral Politics
Incumbency	Electoral Politics
Invalid votes	Electoral Politics
Issue voting	Electoral Politics
Local elections	Electoral Politics
Municipal elections	Electoral Politics
Negative campaign	Electoral Politics
Online campaign	Electoral Politics
Political campaigns	Electoral Politics
Political career	Electoral Politics
Political careers	Electoral Politics
Political competition	Electoral Politics
Political cycle	Electoral Politics
Political preference	Electoral Politics
Political preferences	Electoral Politics
Political recruitment	Electoral Politics
Pre-election surveys	Electoral Politics
Presidential elections	Electoral Politics
Presidential elections 2010	Electoral Politics
Progressive ambition	Electoral Politics
Reelection	Electoral Politics
Re-election	Electoral Politics
Retrospective voting	Electoral Politics
Strategic voting	Electoral Politics
Televised presidential campaigns	Electoral Politics
Vote	Electoral Politics
Vote for federal and state deputies	Electoral Politics
Vote for governors	Electoral Politics
Vote for president	Electoral Politics
Vote for president in Brazil	Electoral Politics
Vote for the Senate	Electoral Politics
Voting	Electoral Politics
Voting behavior	Electoral Politics
Voting machine	Electoral Politics
Young voters	Electoral Politics
Bicameralism	Legislative Politics
Brazilian National Congress	Legislative Politics
Brazilian Senators	Legislative Politics
Chamber of Deputies	Legislative Politics
Coalition	Legislative Politics
Coalition government	Legislative Politics
Coalition presidentialism	Legislative Politics
Coalitions	Legislative Politics
Congress	Legislative Politics
Congressman	Legislative Politics
Constituent Assembly	Legislative Politics
Deputies	Legislative Politics
Executive Legislative relations	Legislative Politics
Executive-Legislative relations	Legislative Politics
Executive-Legislative relationship	Legislative Politics
Federal deputies	Legislative Politics
Federal deputy	Legislative Politics
Federal Senate	Legislative Politics
Formation of cabinets	Legislative Politics
Government coalitions	Legislative Politics
House	Legislative Politics
House of Representatives	Legislative Politics
Impeachment	Legislative Politics
Legislative	Legislative Politics
Legislative Assemblies	Legislative Politics
Legislative behavior	Legislative Politics
Legislative branch	Legislative Politics
Legislative Consulting	Legislative Politics
Legislative debate	Legislative Politics
Legislative elections	Legislative Politics
Legislative elections 2010	Legislative Politics
Legislative forces	Legislative Politics
Legislative information	Legislative Politics
Legislative Power	Legislative Politics
Legislative procedure	Legislative Politics
Legislative process	Legislative Politics
Legislative production	Legislative Politics
Legislative recruitment	Legislative Politics
Legislators’ attitudes	Legislative Politics
Legislature	Legislative Politics
National congress	Legislative Politics
National Congress (Brazil)	Legislative Politics
Parliament	Legislative Politics
Parliamentary Arena	Legislative Politics
Parliamentary behavior	Legislative Politics
Parliamentary career	Legislative Politics
Parliamentary committee	Legislative Politics
Parliamentary elite	Legislative Politics
Party blocs	Legislative Politics
Party coalitions	Legislative Politics
Senate	Legislative Politics
Senator profiles	Legislative Politics
Senators	Legislative Politics
Senators’ profiles	Legislative Politics
State Assemblies	Legislative Politics
State Legislative Assemblies	Legislative Politics
State legislator	Legislative Politics
The Young Parliament	Legislative Politics
Anti-partyism	Political Parties
Brazilian party system	Political Parties
Brazilian Social Democracy Party	Political Parties
Cartel party	Political Parties
Catch-all Party	Political Parties
Media and partisanship	Political Parties
Parties	Political Parties
Partisan bases	Political Parties
Partisan image	Political Parties
Partisan migration	Political Parties
Partisanship	Political Parties
Party constitutions	Political Parties
Party democracy	Political Parties
Party feelings	Political Parties
Party Fund	Political Parties
Party funding	Political Parties
Party identification	Political Parties
Party nationalizations	Political Parties
Party organization	Political Parties
Party politics	Political Parties
Party politics and the environment	Political Parties
Party preference	Political Parties
Party programs	Political Parties
Party strategies	Political Parties
Party system	Political Parties
Party systems	Political Parties
Party unity	Political Parties
Partyism	Political Parties
Political Parties	Political Parties
Political parties and the environment	Political Parties
Political party	Political Parties
Religious partnerships	Political Parties
Small parties	Political Parties
State party systems	Political Parties

B – Paper Maturity Classification

Article Title	Field
¿Por qué la Gente Vota a la Izquierda? Clivajes, Ideología y Voto Retrospectivo en Bolivia y Uruguay en Perspectiva Comparada	E
A Cor dos Eleitos: Determinantes da Sub-Representação Política dos Não-Brancos no Brasil	E
A Dimensão Geográfica das Eleições Brasileiras	E
A Lógica Social do Voto Correto no Brasil	E
À Margem das Margens? A Precisão das Pesquisas Pré-Eleitorais Brasileiras em 2010	E
A Study on the Impact of Campaign Finance, Political Capital and Gender on Electoral Performance	E
Análise da Não-Resposta em Surveys Políticos	E
Candidatos Corruptos São Punidos? Accountability na Eleição Brasileira de 2006	E
Classificando Ocupações Prévias à Entrada na Política: Uma Discussão Metodológica e um Teste Empírico	E
COBERTURA dA IMPRENSA e ELEIÇÕES PRESIDENCIAIS DE 2006 Efeitos Realmente Limitados?	E
Como os Brasileiros Escolhem os Governadores? Desvendando as Razões do Voto para os Executivos Estaduais no Brasil em 2014	E
Determinantes do Voto no Primeiro Turno das Eleições Presidenciais Brasileiras de 2010: Uma Análise Exploratória	E
Dinheiro, Tempo e Memória Eleitoral: Os Mecanismos que Levam ao Voto nas Eleições para Prefeito em 2012	E
Does the Electoral Rule Matter for Political Polarization? The Case of Brazilian Legislative Chambers	E
Economia, Ideologia e Eleições na América Latina	E
Elecciones 2009 en Uruguay: Permanencia de Lealtades Políticas y Accountability Electoral	E
Electoral Governance in Brazil	E
Eleição de Dilma ou Segunda Re-eleição de Lula? Uma Análise Espacial do Pleito de 2010	E
Eleições Municipais Comparadas: A Escolha do Chefe do Executivo no Brasil e no Uruguai e o Impacto Sobre os Sistemas Partidários Locais (2000-2005)	E
Ensino Superior e Voto: Análise do Comportamento Eleitoral da Comunidade Uspiana em 2010	E
Entrando na Arena? Gênero e a Decisão de Concorrer a um Cargo Eletivo	E
Estruturas de Poder nas Redes de Financiamento Político nas Eleições de 2010 no Brasil	E
Fazendo Eleitores e Eleições: Mobilização Política e Democracia no Brasil Pós-Estado Novo	E
Financiamento Eleitoral, Representação Política e Gênero: Uma Análise das Eleições de 2006	E
Grupos Sociais e Preferência Política O Voto Evangélico no Brasil	E
How Important is Twitter to Local Elections in Brazil? A Case Study of Fortaleza City Council	E
Ideologia e Comportamento Político no Eleitorado Brasileiro	E
Impact of Electronic Voting Machines on Blank Votes and Null Votes in Brazilian Elections in 1998	E
INCENTIVOS da DINÂMICA POLÍTICA SOBRE a CORRUPÇÃO Reeleição, Competitividade e Coalizões nos Municípios Brasileiros	E
Investigating Elite Behavior through Field Experiment in Brazil: Do Candidates Answer More to Core or Swing Voters?	E
O Ciclo de Política como Campo Estratégico: O Caso do Benefício de Prestação Continuada	E
O Federalismo como Experiência: Campos Sales e as Tentativas de Estabilização da República	E
Os Custos Eleitorais do Bolsa Família: Reavaliando seu Impacto Sobre a Eleição Presidencial de 2006	E
Os Determinantes da Avaliação da Economia na Eleição Presidencial Brasileira em 2014	E
Partidos, Competição Política e Fraude Eleitoral: A Tônica das Eleições na Primeira República	E
Poder Judiciário e Competição Política: As Eleições de 2010 e a Lei da “Ficha-Limpa”	E
Political Competition and the Diffusion of Conditional Cash Transfers in Brazil	E
Propensão ao Risco e Ambição Progressiva: O Caso dos Deputados Federais nos Estados Unidos	E
PT no Purgatório: Ambivalência Eleitoral no Primeiro Turno das Eleições Presidenciais de 2010	E
The Capital Mistake: Local Information and National Electoral Reforms	E
The Impacts of Conditional Cash Transfers in Four Presidential Elections (2002–2014)	E
TRANSFERÊNCIA de RENDA e ELEIÇÕES A Trajetória Latino-Americana	E
Vale a Pena Ser um Bom Prefeito? Comportamento Eleitoral e Re-eleição no Brasil	E
Vencedores e Perdedores nas Eleições Presidenciais de 2014: O Efeito da Derrota nas Urnas sobre a Satisfação e o Apoio em Relação à Democracia no Brasil	E
Capital Familiar e Carreira Política no Brasil: Gênero, Partido e Região nas Trajetórias para a Câmara dos Deputados	E/L
Corporate Dependence in Brazil’s 2010 Elections for Federal Deputy	E/L
Determinantes dos Padrões de Carreira Política dos Deputados Federais Paulistas entre as Legislaturas 49ª (1991-1995) e 53ª (2007-2011)	E/L
Eleições Presidenciais, Federalismo e Política Social	E/L
Estrutura da Competição pela Presidência e Consolidação do Sistema Partidário no Brasil	E/L
Federalismo, Coalizões de Governo e Escolhas de Carreira dos Deputados Federais	E/L
O Custo da Política Subnacional: A Forma como o Dinheiro é Gasto Importa? Relação entre Receita, Despesas e Sucesso Eleitoral	E/L
O Desempenho das Mulheres nas Eleições Legislativas de 2010 no Rio Grande do Sul	E/L
Os Senadores-Empresários: Recrutamento, Carreira e Partidos Políticos dos Empresários no Senado Brasileiro (1986-2010)	E/L
Perfil Profissional e Distribuição Regional dos Senadores Brasileiros em Dois Séculos de História	E/L
Uma Avaliação Empírica da Competição Eleitoral para a Câmara Federal no Brasil	E/L
Uso Estratégico de Eleições Alternadas? Efeitos da Candidatura para Prefeito sobre a Votação dos Concorrentes ao Cargo de Deputado Federal no Brasil	E/L
Brazilian Congress, 2014 Elections and Governability Challenges	E/L/P
Migração Partidária na Câmara dos Deputados de 1987 a 2009	E/L/P
Nacionalização Partidária e Estratégias Eleitorais no Presidencialismo de Coalizão	E/L/P
A Concentração Eleitoral no Brasil (1994-2014)	E/P
A Estabilidade e a Efetividade da Preferência Partidária no Brasil	E/P
Articulações Intrapartidárias e Desempenho Eleitoral no Brasil	E/P
As Lógicas do Recrutamento Político	E/P
Competição Partidária e Voto nas Eleições Presidenciais no Brasil	E/P
Comportamento Eleitoral e Estratégia Partidária nas Eleições Presidenciais no Brasil (2002 – 2010)	E/P
Contexto e Competição na Política Paulista	E/P
Da Polícia à Política: Explicando o Perfil dos Candidatos das Forças Repressivas de Estado à Câmara dos Deputados	E/P
Diversidade Sexual, Partidos Políticos e Eleições no Brasil Contemporâneo	E/P
Does Size Matter? Electoral Performance of Small Parties in Brazil	E/P
Identificação Partidária e Voto. As Diferenças entre Petistas e Peessedebistas	E/P
Mercado Futuro: A Economia Política da (Re)Partidarização da Imprensa no Brasil	E/P
Nas Brumas do HGPE: A Imagem Partidária nas Campanhas Presidenciais Brasileiras (1989 a 2010)	E/P
Reforma Política no Brasil: Indagações sobre o Impacto no Sistema Partidário e na Representação	E/P
Sentimentos Partidários e Antipetismo: Condicionantes e Covariantes	E/P
Socialismo Moreno, Conservadorismo Pálido? Cor e Recrutamento Partidário em São Paulo e Rio de Janeiro nas Eleições de 2012	E/P
The 2014 Elections and the Brazilian Party System	E/P
Topografia do Brasil Profundo: Votos, Cargos e Alinhamentos nos Municípios Brasileiros	E/P
VOTO ESTRATÉGICO e COORDENAÇÃO ELEITORAL Testando a Lei de Duverger no Brasil	E/P
As Organizações Sociais de Saúde e o Desempenho do SUS: Um Estudo sobre a Atenção Básica em São Paulo	L
Bicameralismo e Poder Executivo no Brasil: Revisão de Projetos Presidenciais entre 1989-2010	L
Comissões Permanentes, Estrutura de Assessoramento e o Problema Informacional na Câmara dos Deputados do Brasil	L
Cooperação, Liderança e Impasse entre o Legislativo e o Executivo na Produção Legislativa do Congresso Nacional do Brasil	L
Crime e Congresso Nacional: Uma Análise da Política Criminal Aprovada de 1989 a 2006	L
Deputados Brasileiros no Twitter: Um Estudo Quantitativo dos Padrões de Adoção e Uso da Ferramenta	L
Determinantes do Controle Horizontal em Parlamentos Reativos: O Caso do Brasil (1988-2005)	L
Facciosismo, Rachas Intrapartidários e Capacidade Decisória na Democracia Brasileira de 1946	L
Financiamento de Campanha e Apoio Parlamentar à Agenda Legislativa da Indústria na Câmara dos Deputados	L
Governo e Oposição no Senado Brasileiro (1989-2010)	L
Governos Minoritários no Presidencialismo Latino-Americano: Determinantes Institucionais e Políticos	L
Impacto da Educação Cívica sobre o Conhecimento Político: A Experiência do Programa Parlamento Jovem de Minas Gerais	L
Influência Majoritária em Eleições Proporcionais: Os Efeitos Presidenciais e Governatoriais sobre as Eleições para a Câmara dos Deputados Brasileira (1994-2010)	L
O Efeito das Emendas ao Orçamento no Comportamento Parlamentar e a Dimensão Temporal: Velhas Teses, Novos Testes	L
O Poder da Caneta: A Medida Provisória no Processo Legislativo Estadual	L
O Poder no Executivo: Uma Análise do Papel da Presidência e dos Ministérios no Presidencialismo de Coalizão Brasileiro (1995-2010)	L
Os Sem-Voto do Legislativo Brasileiro: Quem São os Senadores Suplentes e Quais os Seus Impactos sobre o Processo Legislativo	L
Oversized Government Coalitions in Latin America	L
Parlamentares, Representação Política e Redes Sociais Digitais: Perfis de Uso do Twitter na Câmara dos Deputados	L
Partidos Legislativos e Governo de Coalizão: Controle Horizontal das Políticas Públicas	L
Políticas Públicas e a Descentralização Legislativa da Assembleia Constituinte Brasileira, 1987-1988	L
The Institutional Presidency from a Comparative Perspective: Argentina and Brazil since the 1980s	L
The Legislative Work in an Authoritarian Regime: the Case of the São Paulo Administrative Department	L
Unboxing the Active Role of the Legislative Power in Brazil	L
Variedades de Governos de Coalizão no Presidencialismo. Chile, 1990-2010	L
What is the Best Strategy to Obtain Legislative Support? Survey Evidence from Brazilian State Assemblies	L
Avaliando Vínculos entre Partidos e Deputados nas Assembleias Legislativas Brasileiras	L/P
Senado Brasileiro: Casa Federativa ou Partidária?	L/P
¿Son los Chilenos Igualmente Solidarios?La Influencia de los Recursos Personales en las Donaciones de Dinero	Other
A Afeição dos Cidadãos pelos Políticos Mal-Afamados: Identificando os Perfis Associados à Aceitação do ‘Rouba, mas Faz’ no Brasil	Other
A Comparative Analysis of Brazil’s Foreign Policy Drivers Towards the USA: Comment on Amorim Neto (2011)	Other
A Conciliação e os Meios Alternativos de Solução de Conflitos no Império Brasileiro	Other
A Construção do Campo de Internet e Política: Análise dos Artigos Brasileiros Apresentados entre 2000 e 2014	Other
A Democracia Disjuntiva no Contexto Brasileiro: Algumas Considerações a Partir do Trabalho das Delegacias de Polícia	Other
A Economia Solidária e a Revitalização do Paradigma Cooperativo	Other
A Identificação de Enquadramentos Através da Análise de Correspondências: Um Modelo Analítico Aplicado à Controvérsia das Ações Afirmativas Raciais na Imprensa	Other
A Macrodinâmica Social Brasileira: Mudanças, Continuidades e Desafios	Other
A Política de Segurança Pública no Brasil: Uma Análise dos Gastos Estaduais (1999-2010)	Other
A Psicologia Política das Classes Sociais no Brasil: Atributos das Atitudes Políticas por Estratificação e Mobilidade Social	Other
A Reconfiguração do Sindicalismo Brasileiro nos Anos 2000: As Bases Sociais e o Perfil Político-Ideológico da Conlutas	Other
A Redução da Maioridade Penal Diminui a Violência? Evidências de um Estudo Comparado	Other
A Segregação dos Jovens em Paris	Other
Acesso Individual e Coletivo de Moradores de Favelas à Justiça	Other
Ações Afirmativas no Vestibular da UFPR entre 2005 e 2012: De Política Afirmativa Racial a Política Afirmativa de Gênero	Other
Administrando o Debate Público: O Globo e a Controvérsia em Torno das Cotas Raciais	Other
Aferindo a Capacidade de Influência das Conferências de Políticas Públicas sobre os Programas das Respectivas Políticas Setoriais	Other
Análise Espacial da Burocracia da Assistência Social nos Municípios Brasileiros: Notas para uma Reflexão sobre a Capacidade de Implementação dessa Política	Other
Associativismo e Representação Política Feminina no Brasil	Other
Atitudes, Cognição e Participação Política: Padrões de Influência dos Ambientes de Socialização sobre o Perfil Político dos Jovens	Other
Avaliação de Impacto das Condicionalidades de Educação do Programa Bolsa Família (2005 e 2009)	Other
Beyond Brazilian Coalition Presidentialism: The Appropriation of the Legislative Agenda	Other
Boa Noite, e Boa Sorte: Determinantes da Demissão de Ministros Envolvidos em Escândalos de Corrupção no Primeiro Governo Dilma Rousseff	Other
‘Break-In Parties’ and Changing Patterns of Democracy in Latin America	Other
Cadenas Globales de Valor y Desarrollo: Perspectivas Críticas desde el Sur Global	Other
Capital Social e Políticas Públicas na Região Metropolitana de Porto Alegre: Comparando Novo Hamburgo e Estância Velha	Other
Capital Social y Participación, Una Perspectiva desde el Cono Sur de América: Porto Alegre, Montevideo y Santiago de Chile	Other
Cartografia do Pensamento Político Brasileiro: Conceito, História, Abordagens	Other
Classe Social e Deslocamentos de Renda no Brasil	Other
Classes e Práticas Sociais	Other
Classificando Regimes Políticos Utilizando Análise de Conglomerados	Other
Conciliação e Tensões entre Trabalho e Família para Mulheres Titulares do Programa Bolsa Família	Other
Confiança na Polícia em Minas Gerais: O Efeito da Percepção de Eficiência e do Contato Individual	Other
Confiança nas Forças Armadas Brasileiras: Uma Análise Empírica a Partir dos Dados da Pesquisa Sips – Defesa Nacional	Other
Construindo a América Do Sul: Identidades e Interesses na Formação Discursiva da UNASUL	Other
Cooperação e Coordenação na Implementação de Políticas Públicas: O Caso da Saúde	Other
Cooperation for Development, Brazilian Regional Leadership and Global Protagonism	Other
Coordenação Federativa e a “Armadilha da Decisão Conjunta”: As Comissões de Articulação Intergovernamental das Políticas Sociais no Brasil	Other
Corrupção e Participação Política no Brasil: Diagnósticos e Consequências	Other
Corruption and Political Participation in the Americas and the Caribbean	Other
Cultura Política, Mudanças Econômicas e Democracia Inercial. Uma Análise Pós-Eleições de 2014	Other
Democracy, Redistribution, and Equality	Other
Desempenho Econômico do Rio de Janeiro: Trajetórias Passadas e Perspectivas Futuras	Other
Desenho Institucional e Promoção da Justiça em Espaços Participativos: Implicações em um Estudo de Caso	Other
Desigualdade de Rendimentos do Trabalho no Curto e no Longo Prazo: Tendências de Idade, Período e Coorte	Other
Desigualdades de Gênero na Carreira Acadêmica No Brasil	Other
Diferenciação Institucional e Desigualdades no Ensino Superior	Other
Educação e Rendimentos dos Ricos no Brasil	Other
Empirical Determinants of Government Efficiency: A Study Based on Objective Indicators	Other
Engajamento Cívico e Capital Social: Um Modelo Interativo para o Efeito da Confiança Interpessoal	Other
Entre cooperação e Centralização Federalismo e Políticas Sociais no Brasil Pós-1988	Other
Entre o Altruísmo e o Familismo: A Agenda Parlamentar Feminina e as Políticas Família-Trabalho (Brasil, 2003-2013)	Other
Equidade na Educação Superior no Brasil: Uma Análise Multinomial das Políticas Públicas de Acesso	Other
Escola Pública e Segmentos Populares em um Contexto de Construção Institucional da Democracia	Other
Estado e Desigualdade de Renda no Brasil Fluxos de Rendimentos e Estratificação social	Other
Estrutura de Classes, Educação e Queda da Desigualdade de Renda (2002-2011)	Other
Estrutura Social e Segregação em São Paulo: Transformações na Década de 2000	Other
Explicando os Efeitos de Programas de Socialização Política: A Experiência do Parlamento Jovem no Brasil	Other
Fatores Familiares e Desempenho Escolar: Uma Abordagem Multidimensional	Other
Federalismo e Bons Governos: Uma Análise Política da Gestão Fiscal dos Municípios	Other
Flying in Clear Skies: Technical Arguments Influencing ANAC Regulations	Other
Foreign Aid and the Governance of International Financial Organizations: The Brazilian-Bloc Case in the IMF and the World Bank	Other
Framing Effects on Foreign Policy: Experimental Evidence from Emerging Countries and the Argentine- Brazilian Rivalry	Other
Gênero e Representação Política nos Conselhos Gestores no Brasil	Other
Institucionalização Partidária e Consistência Programática: A Resposta da Esquerda a Ataques Especulativos na América Latina	Other
Integración Sociopolítica y Bienestar Subjetivo en México	Other
Internet and Politics Studies in Brazil: Mapping the Characteristics and Disparities of the Research Field	Other
Juventude por Cor e Renda no Acesso ao Ensino Superior Somando Desvantagens, Multiplicando Desigualdades?	Other
Juventude Violenta: Processos, Retrocessos e Novos Percursos	Other
Liberalism, Marxism and Democratic Theory Revisited: Proposal of a Joint Index of Political and Economic Democracy	Other
Manda Quem Pode: A Elite Governante da Cultura em Portugal (1976-2011)	Other
Mapping and Explaining the Use of the Left-Right Divide	Other
Mecanismos Institucionais de Regulação Federal e seus Resultados nas Políticas de Educação e Saúde	Other
Medindo o Acesso à Justiça Cível no Brasil	Other
Mensurando Alteridades, Estabelecendo Direitos: Práticas e Saberes Governamentais na Criação de Fronteiras Étnicas	Other
Mercantilização no Feminino: A Visibilidade do Trabalho das Mulheres no Brasil	Other
Metamorfoses da Questão Geracional: O Problema da Incorporação dos Jovens na Dinâmica Social	Other
Migrações e Trabalho no Brasil Fatores Étnico-Nacionais e Raciais	Other
Mobilização do Direito como Repertório de Ação Coletiva e Crítica Institucional no Campo Ambiental Brasileiro	Other
Modelando a Participação Social: Uma Análise da Propensão à Inserção em Instituições Participativas, a Partir de Características Socioeconômicas e Políticas	Other
Mortes Prematuras e Conflito Armado pelo Domínio das Favelas no Rio de Janeiro	Other
Mudança de Fronteiras Étnicas e Participação Política de Descendentes de Imigrantes em São Paulo	Other
Mudanças e Continuidades no Movimento de Direitos Humanos: Padrões Organizacionais, Relacionais e Discursivos	Other
O Que Fazem os Conselhos e Quando o Fazem? Padrões Decisórios e o Debate dos Efeitos das Instituições Participativas	Other
O Que Pensa Quem “Bate à Porta” de uma Casa Que Só “Apanha”?: Percepções e Orientações dos Visitantes sobre o Congresso Nacional	Other
O Rio de Janeiro e o Estado Nacional (1946-2010)	Other
O Sistema Oligárquico Representativo da Primeira República	Other
O Topo da Distribuição de Renda no Brasil: Primeiras Estimativas com Dados Tributários e Comparação com Pesquisas Domiciliares (2006-2012)	Other
Omissões e Seletividades da Ciência Política Brasileira: Lacunas Temáticas e seus Problemas Sócio- Epistêmicos	Other
Os Alunos do Ensino Médio e Sciences Po: Entre a Meritocracia e a Percepção das Desigualdades	Other
Os Impactos da Geração de Empregos sobre as Desigualdades de Renda: Uma Análise da Década de 2000	Other
Para Além da Renda: Uma Análise da Pobreza Multidimensional no Brasil	Other
Parcerias entre a Escola e a Comunidade em Portugal: Uma Análise a partir da Avaliação Externa das Escolas	Other
Perceptions on Justice, the Judiciary and Democracy	Other
Personality, Political Attitudes and Participation in Protests: The Direct and Mediated Effects of Psychological Factors on Political Activism	Other
Pesquisa e Produção de Conhecimento Sobre a América Latina na Ciência Política Brasileira	Other
Políticas de Ação Afirmativa e o Experimento de Listas: O Caso das Cotas Raciais na Universidade Brasileira	Other
Pós-Graduação em Relações Internacionais no Brasil: Anotações sobre sua Institucionalização	Other
Posição Socioeconômica, Idade e Condição de Saúde no Brasil	Other
Preto no Branco? Mensuração, Relevância e Concordância Classificatória no País da Incerteza Racial	Other
Privação Relativa e Ativismo em Protestos no Brasil: Uma Investigação sobre o Horizonte do Possível	Other
Proposta de um Indicador Socioeconômico para os Alunos das Escolas Públicas dos Municípios Brasileiros	Other
Protesto Político na América Latina: Tendências Recentes e Determinantes Individuais	Other
Qualificando a Adesão à Democracia: Quão Democráticos são os Democratas Brasileiros?	Other
Quatro Décadas de Mobilidade Social no Brasil	Other
Quem é Classe Média no Brasil? Um Estudo sobre Identidades de Classe	Other
Racionalidade Online: Provimento de Razões em Discussões Virtuais	Other
Renda, Relações Sociais e Felicidade no Brasil	Other
Saúde, Religião e Trânsito Religioso: Estudo Pró-Saúde	Other
Sobre a “Crise” do Estado de Bem-Estar: Retração, Transformação Fáustica ou o Quê?	Other
Sofisticação Política e Opinião Pública no Brasil: Revisitando Hipóteses Clássicas	Other
Solidariedade e Expressão Jurídica: Valores Políticos de Vereadores sobre Direitos Sociais	Other
State Transfers, Taxes and Income Inequality in Brazil	Other
SUPREMO Relator: Processo Decisório e Mudanças na Composição Do STF nos Governos FHC e Lula	Other
Tempo na ou da Justiça Criminal Brasileira: Uma Discussão Metodológica	Other
Território e Desigualdades de Renda em Regiões Metropolitanas do Brasil	Other
The Bigger, the Better: Coalitions in the GATT/WTO	Other
The Elusive New Middle Class in Brazil	Other
The Stratification of Diversity: Measuring the Hierarchy of Brazilian Political Science	Other
The Three Achilles’ Heels of Brazilian Political Science	Other
Transições Ocupacionais e Mercados de Trabalho Intermetropolitanos: Os Casos de Rio de Janeiro e São Paulo	Other
Trazendo de Volta a Mensuração: Fundamentos Metodológicos do Índice de Democracia Eleitoral1	Other
Trust and Political Information: Attitudinal Change in Participants in the Youth Parliament in Brazil	Other
Uma Nova Abordagem Empírica para a Hierarquia de Status no Brasil	Other
Votando na Prateleira: A Politização do Consumo na América Latina	Other
As Transformações nas Formas de Militância no Interior do PT: Maior Inclusão e Menor Intensidade	P
O Eleitor Antipetista: Partidarismo e Avaliação Retrospectiva	P
O Perfil do Financiamento dos Partidos Brasileiros (2006-2012): O Que as Tipologias Dizem?	P
O Sistema Partidário Brasileiro: Um Debate Com a Literatura Recente	P
Política Estadual e Desigualdade: Por Que alguns Estados Redistribuem Mais do Que Outros?	P
Profissionalização Política, Processo Seletivo e Recursos Partidários: Uma Análise da Percepção dos candidatos do PT, PMDB, PSDB e DEM nas Eleições para Deputado Federal de 2010	P
Unidade Partidária e Política Externa na América Latina	P

C – Pre-Analysis Plan

Publication Dates

• Publication in this collection
  14 May 2021
• Date of issue
  2021

History

• Received
  23 Jan 2020
• Reviewed
  5 July 2020
• Accepted
  20 July 2020