Transparency and Replication in Brazilian Political Science: A First Look*1

Transparency and Replication in Brazilian Political Science: A First Look We provide the first replication study of political science research published in Brazil by attempting to replicate every quantitative article published in five major Brazilian journals between 2012 and 2016. We also tested whether replication rates varied between established fields, more traditional and where the use of quantitative data is more common, and emerging fields. Our results show that transparency and reproduction are still in a development stage in Brazilian Political Science. Of the 650 articles reviewed, we asked for data to 197 quantitative articles. From those, only 28% agreed to share datasets and computed codes. We were able to attempt a replication for only 14%, and successfully reproduce the results of less than 5%. We conclude by suggesting the adoption of transparency and replicability procedures that are standard in other scientific communities.


INTRODUCTION
S cientific disciplines are facing a credibility crisis. In social sciences, significant published findings often cannot be replicated (Goodman, Fanelli and Ioannidis, 2016;Key, 2016;Christensen et al., 2019). High-profile publications have been retracted and, in some cases, identified as fraudulent. Some scholars have speculated that at least half of all published research is simply wrong (Ionnadis, 2005). The reproduction of other scholars' work has led to several high-profile reviews and at least one significant retraction of a fraudulent result 1 . All these (and other) problems suggest that much research is a waste In response, scholars have investigated the causes of this credibility crisis and suggested several solutions. The causes include institutional mechanisms that encourage "fishing" or p-hacking 2 (Humphreys, De La Sierra and Van Der Windt, 2013) and lead to selection bias in publication choice (Gerber, Green and Nickerson, 2001). For example, journals are more likely to publish "significant" findings than null results (Ferguson and Heene, 2012). Knowing this, scholars fail to submit null results for publication. They may also hunt through the dataset for significant results. All these mechanisms push relevant null results aside, encourage poor scientific practices, and lead to a high rate of false positives 3 in published studies.
Addressing all these problems will require significant changes in scientific institutions. Scholars have proposed several reforms and best practices that have the potential to deal with these issues. These practices might include open access to academic production, data and research materials, computer codes (preferably in open source formats), open peer review, use of pre-registration and data analysis plans, reviewing research designs instead of results (Cruwell et al., 2019;Martins, 2020), and others promoted by organizations and initiatives, such as the Iniciativa Brasileira de Reprodutibilidade (https://www.reprodutibilidade. bio.br/), the Project TIER (https://www.projecttier.org/), the Berkeley Institute for Transparency in the Social Sciences (https://www.bitss. org/) and the Open Science Foundation (https://osf.io/).
Research transparency is one of these new practices. By research transparency we mean providing all the resources needed for other scholars to replicate one's findings, including source materials, computer code, and other relevant resources (King, Keohane and Verba, 1994;Figueiredo Filho et al., 2019;Bakken, 2019). Transparency permits studies' replication and reproducibility, which are key factors for research's credibility (Janz, 2016). In the same way that transparency is a condition for research reproduction, reproducibility is the ability to reproduce the same results of a study using the same data and methodology (NSF, 2015) 4 .
Adopting transparency and reproducibility procedures would diminish the probability of mistakes (Simonsohn, 2013;King, 1995), increase both the citations rate (Piwowar, Day and Fridsma, 2007) and the trustfulness, efficiency and cumulative advance of academic work (King, 1995;King, 2006;Ball andMedeiros 2012, Finifter 1975;Goodman et al., 2015;Ebersol et al., 2016;Markowetz, 2015;Elman, Kapiszewski and Lupia, 2018;Christensen and Miguel, 2018;Gleditsch, 2020). The transparency and reproducibility culture are starting to emerge in some important fields, such as business and management sciences (Martins, 2020), psychological sciences (Gilbert et al., 2016), biomedical sciences (Bakken, 2019) and social sciences (Christensen, Freese and Miguel, 2019), as well as in great part of developed countries' scientific communities (Degterev, 2020). Some journals already require the release of all replication materials, including data and computer code, as a condition of publication 5 . These requirements have significantly increased the replicability of research in those countries. However, little is known about replicability outside the United States and Western Europe.
Even with all this evidence showing that transparency helps to ensure academic research trustfulness, scientists are, in general, still suspicious about sharing their data with the community (Janz and Freese, 2019) or face many difficulties to guarantee free and usable access to them. The lack of sufficient resources (including technological frames and abilities, but also time and money) to make research data public available is one of the most cited reasons for scientists not to share their data (Tenopir et al., 2011). Besides, there is an important concern about the mistaken use of the shared data, specially as far as plagiarism is concerned (Huang et al., 2012). These reasons help us to understand why the rate of data sharing continues low, evidenced by empirical studies in different scientific fields, like less than 6% in behavioral sciences (Hildebrandt and Prenoveau, 2020), 10-13% in medical sciences (Savage and Vickers, 2009;Thelwall et al., 2020), 14% in psychological sciences (Hardwicke et al., 2020.), 36% in ecology and evolutionary sciences (Roche et al., 2015) and 40% in biological sciences (HUANG et al., 2012).
Using this conceptual framework, we report herein on the first research transparency and reproducibility analysis of political science in Brazil. We attempted to reproduce results of 197 articles reporting quantitative findings published between 2012 e 2016 in the five leading Brazilian political science and general social science journals, the Brazilian Political Science Review, the Revista de Ciência Política, Dados -Revista de Ciências Sociais, Opinião Pública, and Revista Brasileira de Ciências Sociais. Besides examining overall replication rates, we also observed differences in replicability across established, where the use of quantitative data is more common, and emerging subfields, as described below, as well as differences in the availability of data from publications. We tested for differences in methodological approach, transparency, and replicability between research in more and less established subfields in Brazilian Political Science.
Our results show that transparency and reproduction are still in a development stage in Brazilian Political Science. Of the 650 articles reviewed, we asked for data to 197 quantitative articles. From those, only 28% agreed to share datasets and computed codes. We were able to attempt a replication for only 14%, and successfully reproduce the results of less than 5%. Established subfield papers had higher transparency and reproduction rates than those for emerging fields, partly because they were more likely to use quantitative methods. For such articles, we found significantly higher rates of response, data provision, and replication success than with quantitative papers from emerging fields.
This study is divided into four sessions including this introduction. The next session presents the research design that includes the explanation of our hypotheses, criteria for classification of papers into fields (established or emerging) and methodology (quantitative or non-quantitative), details about the request for replication data and coding methodology for our dependent variables. The following session gives details about the replication procedure and the most important results this research has found. The last session is a conclusion of the study and it also brings some suggestions to improve transparency and replicability into the Brazilian social sciences community.

RESEARCH DESIGN
Conducting a replication study is not as trivial as some might think. Replications are difficult to conduct, time-consuming, and hard to publish (Janz, 2016). It is important to follow strict procedures to ensure that its design does not cause a failure in replicating. It seems to be the case of an important replication study conducted and reported by the Open Science Collaboration that attempted to replicate 100 published studies from psychological sciences fields and reported a surprisingly low reproducibility rate 6 . Gilbert et al. (2016) reported that this study's design had errors in sampling, power and biases that would harm its conclusions.
To avoid this kind of error, we used a simpler, but accurate procedure. This study used the same dataset as used in original studies and tried to replicate the reported results without any change in samples or methodologies. We analyzed every paper published in the leading Brazilian Political Science journals, and coded each one according to its methodological approach, response to the data request, and research replicability 7 . We tried to replicate every paper that used a quantitative approach. The focus on "quantitative studies" is simply because such designs should make replication easier. When datasets and computer codes are archived appropriately, other analysts should be able to reproduce every figure or table very quickly. Rich and detailed historical studies may also be replicated, but the effort required to do so for five years of articles was beyond the capacity of our project.

Hypotheses
Our research was intended primarily as a measurement exercise to establish the extent of replication norms in Brazil. However, we also hypothesized that research in established subfields would be more replicable than research in emerging fields, reflecting the same biases discussed above. In established fields, null results are more likely to be published since they may challenge established scholarship. Similarly, the existence of a larger number of faculty focusing on core questions may increase scholars' care in testing hypotheses, knowing others may question their work. In contrast, in emerging fields, manuscripts are less likely to be published unless they have significant results. Consider a study on soccer games and votes for incumbents. If there is a null result, the paper is unlikely to be published and may be dismissed as a bad idea. If there is a significant result, a journal may be inclined to publish a "clever" piece that may get media attention. In this way, emerging fields are more likely to suffer false-positive bias due to the institutional problems discussed above 8 .
We examined all 650 articles published between 2012 and 2016 in the five leading Brazilian political science and general social science journals: Brazilian Political Science Review, Revista Brasileira de Ciência Política, DADOS, Opinião Pública, and Revista Brasileira de Ciências Sociais. Figures 1 and 2 show the distribution of papers by journal and year, respectively.

Figure 1 Papers by Journal
Source: Research data. Elaborated by the authors.

Identifying Article from Established Subfields
To explore our hypotheses regarding mature and emerging areas of study, we first defined four research subfields: Electoral Politics, Legislative Politics, Political Parties, and Emerging/Other. The first three are among the most established not only in Brazilian Political Science, but also in the international academic community. The other categories included all emerging subfields. Then, we collected the keywords from all papers, and we selected the ones related to the established fields. We pre-determined categories based on the paper keywords, as Appendix A shows. From the 180 keywords, 88 (49%) were assigned to Electoral Politics, 58 (32%) to Legislative Politics,and 34 (19%) to Political Parties. In addition, we preregistered these categories at the Open Science Foundation. For example, we categorize keywords like "candidates", "candidacies" and "election" as Electoral Politics; keywords like "House of Representatives", "coalitions" and "Congress" as Legislative Politics and "party system", "party funding" and "partisanship" were included in the Party Politics category.
We classified each paper into one or more of these four categories. From all 650 papers, 154 (24%) used one or more of the selected keywords. Among those, 60 (9%), 37 (6%), and 16 (2%) included keywords from one of the subgroups Electoral Politics (E), Legislative Politics keywords (L), and Political Parties (P) keywords, respectively. An additional 15 (2%) had keywords from both Electoral Politics and Legislative Politics (E/L), 21 (3%) papers included both Electoral Politics and Political Parties keywords (E/P), and two had both Legislative Politics and Political Parties keywords (L/P). Finally, three papers had all three keywords (E/L/P). The pieces that did not have any of the selected keywords were classified in the "Emerging" area of study.
Established subfields represented 23.7% of all 650 papers. The sum of each subfield is not equal to this percentage because a paper can be classified into more than one subfield. Note that the Electoral Studies subfield is slightly more popular than the other two established subfields. Appendix B has a detailed classification of each paper.

Figure 3 Papers by Subfields and Maturity
Source: Research data. Elaborated by the authors.

Identifying Quantitative Works
To implement our research design, we first needed to identify which papers had a quantitative approach that allowed replication using only dataset and computer codes. Our methodological decision to only try to replicate quantitative studies relied on the research's limitations to both analyze and reproduce qualitative data 9 .
We sought the main results reported for each paper and evaluated if it included any quantitative analysis. We created a dummy variable named Quantitative and assumed value 1 if, after a basic overview of the study, we identified quantitative results among the study's main findings. The presence of charts and tables would indicate the use of a quantitative methodology. This selection cannot be considered very strict for a couple of reasons. First, we did the work manually, since there was no way to collect the articles' metadata automatically 10 . Second, in order to enlarge the number of observations, we decided to include studies that had relevant quantitative results but did not necessarily use a statistical inference analytical method. Therefore, we considered a study with only a descriptive analysis as quantitative.
This decision brought up a discussion about differences within the group of quantitative studies. It would be more challenging to replicate research that used complex econometric models than another that did only a descriptive analysis. We are aware of this situation, but, for this research, we chose not to explore this heterogeneity. It would be necessary to develop an objective criterion to classify each quantita-tive methodology by their complexity and separate them into groups according to different replication difficulties. These procedures were beyond our time and work resources.
Even with a soft parameter for Quantitative, the number of quantitative papers was only 197, or 30% of all pieces examined. Notably, the proportion of quantitative articles in established subfields was significantly higher than in emerging subfields. Although they were less than a quarter of the potentially replicable papers, the established subfield papers included more than half of the quantitative research.

Requesting Replication Materials
For each paper reporting quantitative results, we sent an e-mail message to the authors and requested a replication dataset and a computer code. The message explained the research goals and assured author privacy in disclosure of our research, using the model e-mail below: "Subject: Request for Data for Replication Study Without Identification.
Dear Mr(s). (Author's name), We write on behalf of Professors George Avelino (FGV, coordinator of the Center of Politics and Economics of the Public Sector), and Scott Desposato (the University of California, San Diego, director of the Center for Latin American Studies at the same university). We are developing an analysis of the replicability of the Political Science articles published in the leading Brazilian journals in the field, and your paper is in our database.
We would like to request both the dataset and the computer codes required to reproduce the results in your article (Title of the article), published in the journal (Name of the journal) on year (Year), volume (Volume number), (Journal number).
Your name, article name, or any other identifying information about you or your article, will NOT be disclosed in our final report.
Thank you in advance for your contribution to our project. Regards," The procedure followed the schedule below: Phase I (responses) • March 5 th , 2018: sent the original message; • March 26 th , 2018: sent the first follow-up message to those who did not answer the original message and to address any questions; • April 20 th , 2018: sent a second follow-up message to those who did answer the first follow-up message and to answer any remaining questions; • April 27 th , 2018: deadline for receiving answers.
Phase II (completeness) • May 7 th , 2018: sent a second message asking for the complete material for those who answered our first e-mail, but who provided incomplete replication materials; • May 21 st , 2018: final deadline for answers for incomplete material.
In total, scholars had almost eight weeks answer our first message, and 11 weeks to provide replication materials. In general, the ones who actually answered our request did it very shortly. A more extended period might have increased our answer rate, but we received no additional answers after the final deadline.

Coding
Besides the proportion of studies that used a quantitative approach, we examined four transparency and replicability measures applied to each article that we characterized as having some quantitative analysis. Our four measures were: • Responsiveness: Did the author(s) answer our email, even with a refusal to collaborate?
• Agreement: Did the author(s) agree to share data?
• Completeness: Did the author(s) provide enough data and code to enable the research replication attempt?
For Responsiveness, we first coded the papers by combining the answers in the two research phases (the first message and the follow-up) and the lack thereof: 1) Answered in phase 1; 2) Answered in phase 2; 3) Did not answer. Then, we coded Responsiveness as "1" for those that answered our request either in Phase 1 or Phase 2. In total, 83 of the 197 papers (42%) answered our request. They were divided nearly equally between established and emerging fields.
The second variable, Agreement, measures whether the author agreed to share the data or not. The variable had six categories: 1) Answered with data; 2) No response; 3) Data is not available; 4) Cannot share data; 5) Refusal and 6) Other. We coded the variable Agreement as a "1" for articles where the author provided data. This variable was not among our pre-registered dependent variables, but it seemed appropriate to distinguish the authors who answered positively from those who did not. Among the 83 responses, 56 (67.5%) provided some analysis data. From these, 36 papers (almost 65% of the 56) were from established fields.
As far as Completeness was concerned, we coded the following categories: 1) Complete; 2) No useful data; 3) Only code; 4) Only data source, and 5) Only dataset. We coded Completeness as a "1" for articles in which authors sent at least one dataset and some computer code. Only half of the 56 papers that provided data did it completely. More than twothirds of these papers came from established fields (19).
Finally, Replicability had three categories: 1) Completely replicable (all results were reproduced, including sign and statistical significance); 2) Partially replicable (not all results were completely reproduced, or signs/significance did not match the ones reported in the paper) or 3) Non-Replicable (no result could be reproduced). We coded Replicability as "1" for articles in which data that the author(s) provided produced the same results in terms of values, signs, and statistical significance. From those 28 papers with complete data, we were able to fully replicate only 10 (35.7%), seven of which were from established fields. Table 2 presents how dependent variables were coded according to the situations described above. Only studies that were coded "1" for Replicability were considered successfully replicable. Table 3 brings descriptive variables by field type for our sample (quantitative studies) and dependent variables.

Replicability
Author(s) did not answer data request 0 0 0 0 Author(s) answered data request but did not agree to share data 1 0 0 0 Author(s) answered data request and shared data, but data was not complete (dataset or codes missing) 1 1 0 0 Author(s) answered data request and shared complete data but not all reported results were replicable at same statistical significance, value, and sign.
1 1 1 0 Author(s) answered data request and shared complete data but and all reported results were replicable at same statistical significance, value, and sign.

REPLICATION PROCEDURE
We attempted to replicate all papers for which authors shared data and, in some cases, code. The first decision was to define which quantitative results to replicate, since many papers had multiple charts and tables with data not related to the core results. Given the magnitude of the task, we chose to focus on the paper's prominent results. To identify a prominent result, we identified quantitative findings that were most discussed, especially in the abstract, introduction, and conclusion of the study. This procedure constrained our analysis to a smaller number of charts and tables. It also meant we ignored difficulties in replicating secondary analyses.
In replicating, we found problems with data, codes, or results. Figure  4 presents the distribution of these problems. Of the 28 for which we could attempt a replication, we found problems with almost two-thirds of them (18). The most frequent had to do with computer codes, which represented about one-third (10). We also had problems with results and data in eight (28.6%) and seven papers (25%), respectively. Finally, seven articles (21.5%) presented more than one type of problem.

Figure 4 Papers Replicated by Replication Problems
Source: Research data. Elaborated by the authors.
The types of coding problems involved either a non-working or an incomplete statistical computing code. In some cases, we could not calculate a specific result displayed in the paper. In others, there were syntax errors that made a complete replication impossible. We frequently received code lacking one or more calculations, and in many cases, poorly documented, with analyses and data management left unexplained. In one case, the data analysis was computationally intensive and eventually produced results like those published, but it did not reproduce the central figures. Interestingly, many code sets led to results that were not part of the article, suggesting some fishing for results. Finally, there were some minor syntax errors that we were able to correct.
Data problems involved cases in which materials that the authors shared lacked a critical variable or even a key dataset. For example, one author provided a dataset which seemed to be a complete dataset (data frame and code) and which could replicate some of the published regressions. However, further results in the computer code required an additional dataset, which the authors did not share. In some cases, the authors shared a web address where we could find the raw data, but they did not provide specific files with instructions for its coding and cleaning. In other cases, the authors provided a complex code with no instructions on how to execute it.
Some cases were more problematic since we could not open the dataset, or the data differed from the published results. In other cases, authors shared many different files, but none was useful or even required by the accompanying computer code. One author shared a zip file with multiple "xls" files from which we could not replicate results. These papers were classified as "no useful data" and did not proceed to the replication phase.
In some other cases, although we were able to produce results, they did not match those published. We classified those cases as results problems. Some codes failed to define the data subset used for analysis, and the number of observations displayed in the tables did not match with our results. In such cases, even descriptive statistics could not be reproduced. Other papers had problems with the statistical results. For example, our reproduction of regression models yielded different coefficient values for one or more variables.
Data was basically provided in Excel, Stata and R formats, which are well known statistical software. Except for minor corrections in scripts (including adaptations to software updates), replicators were oriented to not modify the code structure. Dataset and code organization is an important part regarding data transparency. In the case an author shared files in a different format or in a way that was not possible to understand how to use it, they were asked to provide more information. In cases that it happened, authors did not answer for our help request and the data were considered as "non-useful".
Successful replications had several shared features. Most notably, the authors provided a well-organized set of data and codes. In one case, the authors did not send the computer code, but their results were clearly described in the text of their article and were straightforward to reproduce. In another case, the authors provided the dataset in a "xls" format file and a Stata file with few lines. Those lines described what table or graph it was related to in the paper. This type of information was critical to enable replication.

Established versus Emerging Quantitative Subfields
Of the 650 papers examined, 197 (30.3%) used a quantitative approach. Of these, only 56 (28.4% of quantitative papers and 8.6% overall) responded positively to our request for data. In half of these cases, the data was incomplete, which did not allow replication. Most of these were cases in which the author shared only codes or only a dataset, but not both. In the end, we completed a replication attempt on 28 papers (14.2% of quantitative papers and 4.3% overall), and only ten of them were fully replicable. In those last cases, the authors provided all data and code to reproduce the core results in the paper. In other words, we were able to successfully replicate only 5% of quantitative and 1.5% of all papers.

(1%)
We hypothesized that established quantitative subfields would have stronger replication norms and higher rates of replicability than emerging fields. We tested our hypothesis by comparing our measures of responsiveness, agreement, data provision, and reproducibility for both established and emerging fields. Table 4 results from T-tests for a difference of means between these two groups among the five dependent variables.
For all our dependent variables, the proportions are higher for the established than for the emerging fields. Authors of research in established areas were nearly twice as likely to respond to our request with some data (36% versus 20.6% in emerging fields). Additionally, 52.8% of authors from those fields provided complete data, versus 45% of those in emerging ones. Finally, of those that provided data, over half of the results from established fields could be replicated (52.8%), versus a slightly lower rate for emerging areas (45%).

Quantitative Responsiveness Agreement Completeness Replicability
Coef.
- The difference of means test presented in Table 3 showed that established subfields papers have a significantly higher probability of adopting a quantitative methodology, answering a request for data, providing the data with more quality, and ultimately being replicable. When including control variables in an OLS regression, the results are similar. Additionally, authors from the same journals were generally less likely to agree to share their data and to share complete data. Finally, results for year of publication have 2012 as baseline and do not strongly suggest an increasing replicability in more recent papers. Only Agreement (2015 and 2016) and Completeness (2016) reported significant higher averages for more recent papers.

CONCLUSION
Around the world, the credibility crisis in science poses a challenge to scholars' efforts to advance knowledge both in the natural and the social disciplines. As Figueiredo et al. (2019) discussed, norms of transparency and replicability are becoming part of standard scientific practice in the United States, Western Europe, and other regions. The scientific community cannot neglect credibility issues anymore. Addressing this crisis will require significant changes in the procedures of scientific institutions, journals, and scholars. There are reforms and best practices that should spread to all academic communities. Especially noteworthy is a general norm of research transparency.
This study is the first look at transparency and replicability in Brazil. It covers only a small sample of the academic work produced in the country and it has its limitations in terms of generalization and comparison with other countries' standards. Even so, the evidence presented here can trigger new research in this field that would help the Brazilian academic community to follow this critical path.
Let us make it clear: our findings do not mean most research is invalid, or conclusions are incorrect. It is just a check on how easy it is to replicate or reproduce the main results. Strictly, if one considers the time lag between submission and publication, our data may even not represent the field accurately during the period 12 . Besides, we only check whether scholars provide the data we requested within two months of our request and whether we can easily use it to generate results identical to those in publication. With more time and more explanation of our purposes, perhaps we would have a higher response rate. With some sources from which we did receive data but failed to replicate results, with more time, maybe we could have reverse engineered findings and asked for more assistance from authors in understanding what they provided. With more time, we could have attempted to replicate results by collecting the original data ourselves from their sources. Again, non-reproducible does not mean invalid in any way. We believe it primarily reflects a lack of transparency norms in Brazilian political science and the need for a new standard according to which it should be easy to replicate and verify any published findings.
The difference in the results from established and emerging fields was also somewhat expected. In general, the more scholars engage in a dialogue around common themes and questions, and the more the field often uses quantitative analysis, the higher their need for transparency procedures to enhance the collective advancement of knowledge.
Admittedly, introducing replication practices in qualitative research poses a different challenge, which would go beyond the scope of this paper. Despite the methodological differences involved, this would be a great improvement for the Brazilian political science community, due to its large number of qualitative works. Also, introducing transparency practices for qualitative research would help to keep pace with the increasing use of mixed methods research designs, which usually requires teams made with members from different methodological backgrounds. Within this picture, trying to define transparency and replication practices in qualitative research, without compromising its small "n" and interpretive characteristics, is a crucial step. (Sukumar and Metoyer, 2019;Moravcsik, 2014aMoravcsik, , 2014bMoravcsik, , 2019.
Our replicability analysis in Brazil suggests that political scientists in that country need to adopt transparency practices. Finally, we offer several recommendations for scholars and journals to improve replicability and maturity in these fields.
Authors can publish a Pre-Analysis Plan or a Registered Report before the beginning of their data collection or analysis (Casey, Glennerster and Miguel, 2012;Miguel et al., 2014), even on qualitative studies (Pineiro and Rosenblatt, 2016). This procedure will strengthen research transparency, mitigating some of the credibility problems we found.
Explicitly specifying the research question and defining our analysis plan will make researchers pay more attention to their study design and focus on theory and research questions. These procedures are relatively easy to adopt, since there are platforms like the Open Science Framework, where one can pre-register research plans and share data, codes, and results.
Scientific institutions can teach and encourage their members to implement these procedures (Steinhardt, 2020). Undergraduate, Graduate and Ph.D programs should have replication classes and should teach the best practices for data management 13 . These classes would increase the students' chances to get their research proposals funded, since a growing number of funding institutions began to require the inclusion of a data management plan in the submitted proposals. For instance, FAPESP, a well-known Brazilian research funding institution, began to require the inclusion of a data management plan (http://www. fapesp.br/gestaodedados/). Additionally, data management courses could also provide an easy way for students to get their first publication by replicating other papers (King, 2006). Finally, a recent paper by Christensen et al. (2019) estimates that papers that make data and coding available receive additional scholarly citations.
Journals can support the publication of null results and of review processes that focus on research questions and design, rather than provocative results. For example, the Center for Open Science recommends that journals include peer review of research plans as well as results, known as Registered Reports: Registered Reports is a publishing format used by over 200 journals that emphasizes the importance of the research question and the quality of methodology by conducting peer-review before data collection. High-qua-lity protocols are then provisionally accepted for publication if the authors follow through with the registered methods. This format rewards best practices in adhering to the hypothetical-deductive model of the scientific method. It eliminates a variety of questionable research practices, including low statistical power, selective reporting of results, and publication bias, while allowing complete flexibility to report serendipitous findings. (https://cos.io/rr/) Registering a research plan is not an instrument to restrict scientists on their flexibility to conduct their research or a safeguard for journals against their responsibility for the credibility of their publications. Unpredictable events may occur during data collection, imposing changes in analysis strategy. A pre-registered report informs readers about such unforeseen problems faced by researchers, and solutions implemented, enhancing research transparency (Christensen, Freese and Miguel, 2019). In other scenarios, readers never learn the real reason behind the methodological choice, opening the way for credibility questioning. For journals, pre-registered analysis plans are an opportunity to mitigate publication bias and reduce incentives for fishing and p-hacking.
Officially, releasing replication data is another essential practice to improve science transparency. Using and sharing computer codes allow research verification and replication (King, 2003;Elman, Kapiszewski and Lupia, 2018;Dafoe, 2014). Publishing replication materials increases incentives for careful analysis and accurate reporting. Verification also allows for investigating potential cases of fraud. Indeed, it would have been impossible to uncover recent cases of data fabrication in the United States scandals without access to underlying data. Replication materials can also support academic teaching and learning.
To keep in touch with the international trend, the Brazilian scientific community should incorporate transparency practices. ABCP (Associação Brasileira de Ciência Política), as well as other associations and Journals, should adopt standards of replication, requiring and archiving data and replication code for published research, subject to the privacy and copyright restrictions.
They could endorse or even implement together a general data repository to concentrate all scientific work. A well-known benchmark is The Berkeley Initiative for Transparency in the Social Sciences (BITSS).

The Berkeley Initiative for Transparency in the Social Sciences (BITSS)
is another new institution that has emerged in recent years to promote dialogue and build consensus around transparency practices. BITSS has established an active training program for the next generation of economists and other social scientists, as well as an award to recognize emerging leaders in this area, the Leamer-Rosenthal Prize for Open Social Science (Christensen and Miguel, 2018:61-62).
Adopting these recommendations will increase the maturity and credibility of social science in Brazil and benefit scholarship and society. Associations and journals can both encourage and facilitate such practices. For example, ABCP and universities can offer courses to teach their members how to track their research procedures, how to prepare a dataset, and how to use computer programs that facilitate this task. In addition, to further transparency, ABCP could lead an association with an internationally known data storage, such as Dataverse (https:// dataverse.org/), which could increase the rewards for registering dataset and codes by making them available to the international scientific community. Universities need to nurture a reproducibility and replication culture within their departments and courses to ensure that a "gold standard" of credibility in academia is strengthened (Janz, 2016). They can also provide online storage for datasets and code sharing.
To further the publication of datasets that interest a broad audience, journals' editors can create a section dedicated to papers presenting datasets to the community. Journals can also publicly require datasets and codes to submission approval (Dafoe, 2014). Finally, journals need to encourage replication publications, but not implement a reverse publication bias, focusing only on failed replications attempts (Janz and Freese, 2019). Recently, the Brazilian Political Science Review adopted a policy that supported transparency. From their author's guideline for submission: Authors submitting empirical papers should provide, when and if the manuscript is approved, the database used in the analysis, the code dictionary describing the variables, and the code for replication or the series of steps of the analysis. The database must be in an easily accessible format for conventional statistical software -such as R, Stata, SPSS, and Excel. The code dictionary should indicate the name of the variable, its description, and the source of the data. Where appropriate, the code for replication of the analyzes should be extensively commented. When the article does not use statistical software that allows the creation of a replication code, the author should describe the procedures, step-by-step, for replication to be possible.
In a good note, a more recent survey with the editors of all other four journals investigated in this research shows they are all intending to introduce transparency and replicability procedures soon. This result shows that concerns with those procedures are starting to prevail in the Brazilian Political Science, making it closer to the international community. A future analysis will tell whether this is an excessive optimistic vision or not.
The scientific community is increasingly adopting transparency norms. Most likely, they will become mandatory for international publication and international research funding. The sooner Brazil's scientific community incorporates these transparency standards, the better. In short, implement transparency procedures -like research pre-registration and sharing of codes and data -is beneficial for everyone. It mitigates credibility suspicions, values scientific work, and, above all, allows research activity to concentrate on what is most important: the questions they seek to answer.
(Received on January 23, 2020) (Resubmitted on July 5, 2020) (Accepted on July 20, 2020) 7. Since we covered the five leading journals for five years, increasing either number of journals or years should not make a difference for our analysis.

NOTES
8. This example is provided for illustration purposes only and is not intended to comment on any research project.
9. For more details on the challenges of implementing reproduction and replication of qualitative studies see Elman and Kapiszewski (2014), Tuval-Mashiach (2017)

Transparency and Replication in Brazilian Political Science: A First Look
We provide the first replication study of political science research published in Brazil by attempting to replicate every quantitative article published in five major Brazilian journals between 2012 and 2016. We also tested whether replication rates varied between established fields, more traditional and where the use of quantitative data is more common, and emerging fields. Our results show that transparency and reproduction are still in a development stage in Brazilian Political Science. Of the 650 articles reviewed, we asked for data to 197 quantitative articles. From those, only 28% agreed to share datasets and computed codes. We were able to attempt a replication for only 14%, and successfully reproduce the results of less than 5%. We conclude by suggesting the adoption of transparency and replicability procedures that are standard in other scientific communities.
Keywords: replication; transparency; reproduction; Brazilian Political Science; data sharing