Transparency and accountability of government algorithms: the case of the Brazilian electronic voting system

In recent years a number of open data movements have emerged around the world, ensuring citizens more opportunities to access information, transparency being a factor associated with trust in public organizations and government. The transparency in algorithms translates into the knowledge of the steps performed and the criteria adopted to obtain a certain result. The objective of this study is to identify the characteristics of transparency and accountability of the Brazilian electronic voting system. Through the case study, recommendations and good practices of transparency in algorithms were examined with the particularities of the Brazilian electronic voting system, which provides control and oversight to society. This study advances the discussion on the influence of new technologies on democracy, placing the Brazilian electronic voting system within the limits of transparency and accountability of algorithms and the government in general.


INTRODUCTION
In recent years, a series of open data movements have appeared around the world (ATTARD, ORLANDI, SCERRI et al., 2015). Citizens have had more opportunities to search for and access information directly from the government, with transparency being a factor associated with the population's trust in public organizations and the government (ALBU and FLYVERBOM, 2016).
Within this context, transparency is understood to be making information available, which permits external actors to monitor performance, from the perspective of citizen participation in the process, and as a result, their capacity to evaluate the actions of authorities (GRIMMELIKHUIJSEN and WELCH, 2012). Even though there is no consensus in these approaches in terms of what transparency is, this field of research agrees that it involves the extent to which stakeholders have access and a similar understanding of what information is necessary without loss, noise, delay or distortion (PAPENFUSS and SCHAEFER, 2015). Accountability consists of the duty of an individual or organization to be responsible in some way for the way they conduct their actions (HOOD, 2010). Transparency and accountability are constructs which are closely linked, with transparency being one of the components of the process of accountability in the public sector (VASCONCELLOS, LUNKES and TALIANI, 2018).
Recent discussions have focused on the transparency of algorithms which are being used to make decisions and are being incorporated into public systems, such as transport, health and policing (FINK, 2018). Algorithms consist of a structured group of information system commands, designed to process instructions and information to generate a result, with it being true that in information societies, the activities, choices and decisions previously decided by humans are now being more and more often handled by algorithms, which can advise and sometimes decide how data should be interpreted and what actions to adopt (MITTELSTADT, ALLO, TADDEO et al., 2016).
Algorithms have become more and more autonomous and invisible, making it difficult for the public to examine and identify their impartiality, and sometimes they introduce inadvertent biases, reinforcing historic discrimination, favoring a political orientation or reinforcing undesirable practices (JANSSEN and KUK, 2016). According to the World Wide Web Foundation (2017, p. 16), making an algorithm more accountable is "making sure that data can be evaluated, controlled and corrected." Fink's studies (2018) about the transparency of the algorithms used by the American government indicate that its policies and practices related to the publishing of algorithms are inconsistent, with there being a need for more effective mechanisms to promote accountability and future studies which will consider the transparency of algorithms in the debate about the limits of accountability and the government in general. Ananny and Crawford (2018) identify limits to transparency and propose a starting point for a algorithmic accountability model. It should be noted that there is a prevalence of theoretical studies in this field (MACHADO, 2018), which reveals the importance of empirical research about algorithmic transparency.
The purpose of this study consists in identifying the characteristics of transparency and accountability in the Brazilian electronic voting system, which has been considered innovative in applying information and communications technology to public management (BALBE, 2014).
In this article we contextualize the importance of this subject in modern information societies from the perspective of new technologies and democracy, digital governance, and algorithmic transparency and accountability. This is followed by a description of this study's methodological steps. Then the model and proposed analysis dimensions are presented and compared with the particular characteristics of the Brazilian electronic voting system. Finally, the results of this case study are presented, and an agenda for future research regarding algorithmic transparency and accountability in the public sector is suggested.

Digital Governance
New technologies have the potential to construct a new relationship between governments and citizens, making possible more efficient, democratic and transparent public administration (GUIMARÃES and MEDEIROS, 2005).
In this sense, the governance of the public sector can be defined, according to Lane (2000), as a group of theories about how governments articulate themselves to provide services to society. Public governance has been understood as a process through which institutions, organizations and citizens guide themselves, treating the interaction between the public sector and society and how they organize themselves to make collective decisions, in order to provide transparent mechanisms so that these decisions materialize (UNITED NATIONS, 2002).

Transparency and accountability
Transparency consists of the availability of information on an organization, which enables external actors to monitor its internal functioning or performance, which is more and more seen as a mark of good governance (RUIJER and MEIJER, 2016). Transparency is considered to be a condition of accountability -which is frequently associated with the concept of responsibility -and a fundamental requirement for good governance (GOEDE and NEUWIRTH, 2014).
The term accountability has been used in the American literature and a large portion of Brazilian literature to address issues related to displaying the accounts of those who govern and their democratic responsibilities (PRADO, 2009). Abrucio and Loureiro (2004) define democratic accountability as a construct of institutional mechanisms by which those who govern are held responsible for their acts by those they govern.
According to the United Nations, accountability is one of the prerequisites for democracy and good government in which those who govern and public servants are responsible for their actions and decisions. Accountability identifies who is responsible for what and what type of conduct is illegal (UNITED NATIONS, 2006). Raupp and Pinho (2013) point out that the concept of accountability can be perceived in light of the dimensions of transparency, participation and the presenting of accounts.
Transparency can be understood as a rule of conduct to be followed by public agents, so that their actions are open to society making social control by citizens and institutions possible (FOX, 2007). Grimmelikhuijsen and Welch (2012) propose a broad definition of transparency which considers the relationship between the State and its citizens and makes possible the evaluation of state activities. Ruediger (2003) assesses the importance of social participation affirming that the democratic potential of new technologies depends on greater demands by civil society for transparency and participation. Akutsu and Pinho (2002, p. 732) state that "[...] without organized civil society, public managers will not feel obliged to promote accountability." Accountability involves "[...] (objective and subjective) responsibility, control, transparency, the obligation to present accounts, justifications for actions which were or were not taken, rewards and/or punishment" (RAUPP and PINHO, 2014, p. 145). To Mainwaring and Welna (2003), accountability or the presenting of accounts by a public agent possesses three main elements: (i) answerability, which is transparency understood as making information available, (ii) responsiveness, understood as responding to demands for information and taking responsibility for these respective acts, and (iii) enforcement, the capacity of sanctions and coercion to ensure access to information. Meijer (2014) teaches us that the instrumental value of transparency is frequently related to accountability, serving as a prerequisite for the information phase of any of its processes. In this particular, authors such as Fox (2007), Hood (2010) and Meijer (2014) propose specific analysis models to study the relationship between transparency and accountability.
In the Brazilian context, Campos (1990), in a seminal work, pointed out the difficulty of translating the concept of accountability to Portuguese. She related the need to protect citizens from bad bureaucratic conduct and found that the absence of public accountability mechanisms is due to the not very democratic relationship that exists between the State and society. Pinho and Sacramento (2009), revisiting Campos's text, emphasize that the concept of accountability involves responsibility, control, transparency, and the obligation to present accounts. They conclude that an institutional and cultural evolution can be perceived which is in favor of increased accountability. Loureiro, Teixeira and Prado (2008) indicate the relevance of empirical studies of governmental transparency associated with the debate about accountability, in the context of constructing and improving democratic institutions.
In this study, we will adopt the scope of the definition proposed by Hood (2010): accountability consists of the duty of an individual or organization to respond in some way in regard to the manner in which they conduct their actions.

Transparency and Accountability of Governmental Algorithms
Algorithms are more and more present in governmental operations and, as a result, in governmental decisions, making them more effective and, at the same time, keeping information secret in "black boxes" (PASQUALE, 2015). Kroll (2015) states that important decisions about people are being taken by algorithms: the counting of votes, voter rolls are purged, the concession of financial assistance, the selection of tax payers for auditing, of passengers to be inspected and even the granting of credit.
Algorithms are not simple and objective instructions (WILLSON, 2017); they are based on society's perceptions and understandings and not on tangible, global and fixed laws (EISCHEN, 2003). Algorithms have biases that can reflect discriminatory practices in society, due to the limitations of computational systems which may be perceived only after interaction with users (FRIEDMAN and NISSENBAUM, 1996).
Algorithms or some of their inputs may be secret, their implementation may be secret or the process may not be precisely described, which will make it difficult to monitor if a citizen or authority suspects that something is wrong and wishes to verify whether decisions are being made in accordance with current policy. Citizens and society have an interest in making these processes more transparent, given that bases for making these decisions are rarely made available to the public (KROLL, 2015).
The algorithms which support the electronic voting process can be divided into three groups: pre-voting, voting and post-voting. In the first, algorithms provide validation of electors who are apt to vote, as well as the validation of registered candidates, and the verification and guarantee of inviolability of electronic ballot boxes. In the second, algorithmic instructions validate the identity and credentials of the voter and count and store votes in a way that ensures and respects their secrecy. Finally, algorithms need to verify whether the ballot boxes have been adulterated, comparing registered votes with voting registers and the data load of each ballot box, permitting the recounting of votes when necessary, guaranteeing the integrity of voting in case of the need to switch ballot boxes, totaling the results and realizing the calculations inherent in voting legislation such as proportional representation (STOICA and GHILIC-MICU, 2016).
Thus, it is fundamental to make algorithms more transparent for the community in general, including their developers, users and those affected by their results, in order to identify and deal with biases, with it being true that this material gains relevance to the extent that algorithms are modified and become more complex, concentrating power in the hands of the people who understand their functioning (JANSSEN and KUK, 2016).
Algorithmic standardization has incited a series of preoccupations about responsibility, impartiality and the autonomy of this process, and this has been aggravated by the rigidity of computational systems responsible for a large variety of decisions in the lives of people (ZIEWITZ, 2015), who tend to accept that automated processes are true and precise (CITRON, 2007).
Transparency can correct the errors in any algorithmic process, promoting efficiency and allowing individuals to correct imprecise data collected about them. In this way, transparency also brings scrutiny which pressures agencies to improve their practices, so that transparent processes will be more precise, and therefore, efficient (ZARSKY, 2016).

Judith Donath, of Harvard University's Berkman Klein Center for Internet & Society, laments the lack of transparency in systems and warns:
The danger of greater dependence on algorithms is that the decision-making process has become oracular: opaque and unquestionable. The solution is design. The process should not be a black box into which we feed data and an answer comes out, but rather a transparent process projected to not only produce a result, but to explain how it arrived at this result. Systems should be capable of producing clear and legible texts and graphics that can help users -readers, editors, doctors, patients, loan requesters, voters, etc. -understand how the decision was taken (ANDERSON and RAINIE, 2017, p. 22).
Besides providing access to information by being transparent, organizations need to guarantee that citizens understand what is happening and how this information is generated and used (NUNES, CAPPELLI and RALHA, 2017).
The concept of algorithmic accountability began to appear due to the possibility of the damage and discrimination that they can cause. The concept of accountability discussed above, when applied to algorithms, has often been confused with transparency. Transparency is an essential component of accountability, permitting citizens, consumers, journalists, monitoring organizations to verify and understand inputs, processes and outputs of a complex algorithmic system to identify evidence of damages and make provisions for reparations if necessary (DIAKOPOULOS, 2014). According to the World Wide Web Foundation (2017, p. 16), making an algorithm more accountable "[...] means guaranteeing that the damage can be evaluated, controlled and corrected."

Research Method
This qualitative study is guided by the case study method (YIN, 2001), as a way to investigate the phenomenon of the interaction between new technologies and democracy, including the characteristics of transparency and accountability in the Brazilian electronic voting system.
The selection of the Brazilian electronic voting system as a case to be studied is due to the relevance of the public service involved -national and regional elections -as well as the interest of the community in perceiving the transparency of the system and monitoring how fairly it functions.
New information and communications technologies have led to the appearance of so-called electronic democracy, or e-democracy, whose most important manifestation is the electronic voting system (FREITAS and MACADAR, 2017). The Brazilian electronic ballot box has been considered an innovative experience in applying information and communications technology to public management (BALBE, 2014). However, the lack of transparency in the voting system is often reflected in the voter's lack of confidence in the results of the voting (ACHIENG and RUHODE, 2013).
Our data collection was based on three sources, in order to trace a convergent line of research among a variety of collected pieces of evidence (YIN, 2001). The first refers to the perception and reports of specialists about the electronic voting system, collected from interviews granted to press outlets. The second consists of documental analysis -scientific articles, and reports and documents by governmental and non-governmental organizations, focused especially on transparency and accountability within the context of digital governance in the public sector. Finally, we used documental evidence from Brazilian voting legislation, which regulates the studied analysis unit.
In this way we employed the triangulation strategy, gathering information from distinct sources of evidence, in order to establish the convergence of our findings (FLICK, 2009;YIN, 2001). The triangulation of data sources is effective when data collection occurs regarding the same aspect of a phenomenon, seeking convergences and divergences (BRUNING, GODRI and TAKAHASHI, 2018)bem como atribuir maior confiabilidade e fidedignidade aos Estudos de Caso e discutem-se cinco modalidades de triangulação: (i. In this manner, the use of multiple sources assures greater credibility and reduces the possibility of research bias (EISENHARDT, 1989).
Box 1 consolidates the cast of specialists (three linked to the Electoral Justice system and three external actors) considering perceptions and reports, in interviews granted to the press, which deal with the electronic voting system and its characteristics, especially in terms of aspects of transparency, security and reliability. The selection of scientific articles was performed in three phases. The search, conducted in November 2018, was based on directories available through CAPES and Google Scholar, using the keywords "e-voting", "algorithm", "transparency", "public services" and "accountability" in the title, subject or keywords, without restriction in terms of the year of publication since this is a recent subject. We identified 3,760 studies. The selection was refined taking into consideration works related to the transparency and accountability of new technologies applied to the public sector.

Box 1 Cast of Specialists
In addition, we made Google searches using the same keywords, which resulted in the selection of documents published by organizations dedicated to the study of algorithmic transparency in systems in general ( The content analysis of these documents led to the construction of the constant analysis model of Figure 1, which contains seven dimensions described in Box 2.

Figure 1 Analysis Model of Algorithmic Transparency and Accountability
Source: Elaborated by the authors. With the documental analysis of Brazilian legislation which regulates the national policy of digital governance (BRASIL, 2016a(BRASIL, , 2018 and the electoral regulations which govern the electronic voting process (TSE, 2018), we identified and categorized the practices adopted by the Brazilian voting system which tend to make it more transparent and accountable. The possibility of analyzing the laws, decrees and resolutions for this type of research was indicated by May (2004). Richardson (2012, p. 228) observes that documental research "[...] does not have the objective of social phenomena, in terms of when and how they are produced, but rather the manifestations which register these phenomena and the ideals elaborated by them. Similar to content analysis, documental research requires the codification of information and the establishment of categories." The practices and characteristics of the transparency and accountability of the Brazilian electronic voting system were analyzed and categorized in light of the analysis model described in Figure 1, resulting in the consolidation presented in Box 4. We consider that the system practices or characteristics which are linked to the item of analysis have been met, and those that deal with one or more aspects of the item are considered to have been partially met. The non-existence of a practice related to the analyzed item was considered unmet. Finally, the evaluation items that are not linked to the evaluated unit of analysis (the Brazilian electronic voting system) were treated as "not applicable." The data analysis was based on technique of content analysis (BARDIN, 2016), pre-analyzing the documents to select relevant aspects, and analyzing them using inference and interpretation, characterizing the most significant thematic elements.

RESULTS AND DISCUSSION
In this section we will discuss our case study results in light of the analysis model displayed in Figure 1, whose dimensions are summarized in Box 2. The evaluation of the algorithms and systems used in digital public services with the assistance of the analysis model makes it possible to verify whether the analyzed object displays the expected behavior, free from errors, biases or discrimination.

Dimension
Description Source

Consciousness
Information should be made available to interested parties (owners, designers, developers and users) to be conscious of possible errors involved in the project, its implementation and use, as well as potential damage which biases can cause to individuals and society.

Access and correction
Adoption of mechanisms which permit questioning and reparations for individuals and groups which are harmed by decisions that receive information from algorithms.

Data origin, privacy and justice
Taking care that the algorithms are not trained with biased data, taking the opportunity to correct them. Preoccupations with privacy, and the protection of commercial secrets or the revelation of the logic which can permit that people with bad intentions can make the system vulnerable, can justify restricting access to qualified and authorized individuals.
Box 3 summarizes, with the data consolidated in Box 4, the dimensions, evaluated items and the degree of adhesion to the Brazilian electronic voting system analysis model. The analysis of Box 3 makes it possible to monitor the degree of adherence to best practices of algorithmic transparency by dimension, as high (A≥75%), average (50%≤A<75%) or low (A<50%).

Dimension
Nº In the "consciousness" dimension we verified a high degree of adherence (100%) in the case evaluated by the analysis model, reflecting the preoccupation of the Electoral Justice system with promoting advertising campaigns to strengthen society's trust in the electronic voting system (TSE, 2017a).
The "accountability", "explanation" and "data origin, privacy and justice" dimensions have low degrees of total adherence (25%, 33% and 33%, respectively) to the analyzed items.
The efforts undertaken by the Electoral Justice system in publishing the functioning of the system, perceived in the consciousness dimension, contrast with the low adherence encountered in the accountability dimension. In fact, the difficulty that citizens have in understanding and inspecting the state's actions in using the electronic voting system has led to the persistence of distrust in terms of its fair functioning, even though more than 20 years have passed since it was first developed.
The "access and correction", "auditing" and "validation, precision and test" dimensions have average degrees of adherence (66% and 50%, respectively) to the analyzed items.
Box 4 presents the final configuration of the dimensions and their respective items of analysis contrasted with evidence of transparency identified in the Brazilian electronic voting system.

Dimensions of Algorithmic Transparency and Accountability
Practices of the Brazilian Electronic Voting System Degree of Adherence A 7.2. Identifies and stores sources of error and uncertainty in the entire algorithm and its data sources, in order to provide information in mitigation procedures.
Programs utilized by the Electoral Justice system are presented for inspection in the form of source programs and executables. A 7.3. Regularly reviews algorithms which orient significant decisions in order to assure the achievement of objectives and avoid adverse effects.
Auditing of the systems occurs through so-called parallel voting, in which ballot boxes are randomly selected (on the eve of the election) to verify that they comply with specifications.

AP
Source: Elaborated by the Authors.
In the "consciousness" dimension, Electoral Justice initiatives were identified in the promotion of advertising campaigns (TSE, 2020), in order to clarify for users the possibility of biases and potential damage caused by the system (ACM, 2017). The limitations of the system in terms of its functionalities (STATS NZ, 2018) are published by the Electoral Justice system showing the restrictions to voting in traffic (possible in cities with more than 100 thousand voters and outside of the State just for presidential elections) and locations where it is only possible to vote with the collection of biometric data. The delivery of an effective public benefit, prioritizing the needs of society (BRASIL, 2016b;STATS NZ, 2018), is emphasized by the Electoral Justice system, which points out that the use of information technology in the electoral process has made it possible to eliminate various types of fraud and human error that existed in the old electoral process, such as the duplication of votes and the switching of ballots during the counting, and has also saved time, energy and resources, and has impeded adulterations and guaranteed voting secrecy (BRASIL, 2016c).
In the "access and reparations" dimension, public agencies should encourage the adoption of mechanisms which permit questioning and correction by those who have been negatively affected by decisions based on information provided by 707-712 Cad. EBAPE.BR, v. 18, Special Edition, Rio de Janeiro, Nov. 2020.
Transparency and accountability of government algorithms: the case of the Brazilian electronic voting system Douglas Morgan Fullin Saldanha Marcela Barbosa da Silva algorithms (ACM, 2017;ALMEIDA and DONEDA, 2016). In this sense, the practice of publishing ballot box bulletins on the internet has been verified (TSE, 2017c), which makes it possible to contest any result sent to the Supreme Electoral Court which diverges from the original bulletin. In addition, the digital records of votes (RDV) make it possible to recount votes in an automated manner (BRASIL, 2016c, p. 28).
It can also recommend ways to pursue reparations for adverse effects, providing contacts so that the user knows how to proceed in these cases and the designation of a person responsible for the social impact of algorithmic decision-making systems (ALMEIDA and DONEDA, 2016;DIAKOPOULOS, FRIEDLER, ARENAS et al., 2016). In the electronic voting system, it is the Electoral Justice system that resolves any conflicts.
It should also have a contingency plan in case of unexpected behavior by the system (DIAKOPOULOS, FRIEDLER, ARENAS et al., 2016). In this case, the Electoral Justice system envisions the adoption of procedures which seek to correct problems presented in electronic ballot boxes during voting, counting or operational errors committed by table volunteers. These procedures are known as voting and counting contingency plans and they are not always capable of resolving all situations, which may lead to the possibility of totally manual voting (with the utilization of ballots and a conventional ballot box) or mixed voting (part electronic and part manual) (TSE, 2019c).
In the accountability dimension, the government has to provide timely and reliable information -i.e. the publication of the system's source code (DIAKOPOULOS, FRIEDLER, ARENAS et al., 2016) -so that the citizen can supervise state action (BRASIL, 2018) and the public manager will be responsible for the decisions made by the algorithms (ACM, 2017). In the electronic voting system, the source programs and executables are subject to inspection (TSE, 2017a). In addition, the effectiveness of the dimension under analysis may be strengthened by collaboration with citizens in all of the cycle phases of public policies and in the creation and improvement of public services (BRASIL, 2018). It is relevant to point out that a large portion of the security and transparency mechanisms incorporated in electronic voting systems are the fruit of technical recommendations made by civil society entitites and suggestions from party bodies (CUNHA, 2009).
Bokslag and Vries (2016) affirm that the electronic voting process is much less transparent, especially for lay people. This is because it's necessary to have an advanced knowledge of cryptography to prove that your vote was taken into consideration in the election results and that all of the votes were counted correctly. Just a small number of researchers will understand this, while the rest of the population will have to trust a system that they cannot understand.
Janino (2017), the Supreme Electoral Court's Secretary of Information Technology, attributes the distrust of the Brazilian population with electronic voting to the population's low level of education, which makes people incapable of understanding the underlying technology of the system. The specialist Rezende (2018) argues that only a printed ballot can give the voter a way to trust the system itself, without needing to believe the words of a technology specialist. This lack of understanding corroborates the low degree of adherence in the accountability dimension.
In the "explanation" dimension, it is understood that systems and institutions that use algorithmic decision making should provide explanations of the procedures followed (ACM, 2017) in technical terms (STATS NZ, 2018) and using common language (DIAKOPOULOS, FRIEDLER, ARENAS et al., 2016), which should include examples and infographics (STATS NZ, 2018). In this requisite, it has been verified that the Supreme Electoral Court has a variety of explanatory material about the electronic ballot box (TSE, 2019d), and electoral statistics (TSE, 2019e), in addition to making available technical documents such as digital summaries (hashes) which are generated in an electoral system sealing ceremony (TSE, 2019f). Icaza (2018), director of the Department of Electoral Cooperation and Observation of the OAS, believes that the Supreme Electoral Court's explanations of the functioning of the electronic ballot boxes in the 2018 elections occurred with "total and absolute transparency".
The "data origin, privacy and justice" dimension contemplates procedures to avoid the manipulation of the system by malicious actors, impeding discriminating effects, and protecting citizen secrecy and privacy, purpose for which the data was collected. If the process is not transparent, it is less likely that people will trust the decisions made and will not continue to share their personal information. (STATS NZ, 2018). In order to avoid malicious actors manipulating the system, only accredited entities and institutions can have access to the software code used by the Electoral Justice system (TSE, 2017a). The electronic ballot box should assure the secrecy and inviolability of the vote (BRASIL, 1997), with security being reinforced by the inexistence of a connection with the internet (BRASIL, 2016c). Almeida (2018) believes that it cannot be affirmed that the electronic ballot box is 100% reliable, but it possesses security mechanisms that are far superior to those involved in the manipulation of paper.
The "auditing" dimension consists of the possibility of verifying corrections of the algorithm inserted in the digital service.
To make this feasible, models, data and decisions should be stored for later analysis, in case there is a suspicion of damage (ACM, 2017). It should also allow the research community to perform public audits (DIAKOPOULOS, FRIEDLER, ARENAS et al., 2016). Finally, it is recommended that decisions informed by algorithms have human supervision, and be preferentially performed by various interested actors (BRASIL, 2018) and supervised by governance groups to ensure the meeting of standards of ethics and privacy (STATS NZ, 2018). The ability to audit an electronic voting system is related to transparency and is an important in terms of the way the system is perceived by the public at large (FREITAS e MACADAR, 2017).
In the dimension under analysis, we have identified that entities and institutions accredited by the Supreme Electoral Court can monitor and audit the software used in Brazilian elections (TSE, 2017c). The system is submitted to public security tests which various entities can participate in, including university departments of Information Technology (TSE, 2017a). The elections are supervised in each location by electoral boards which consist of provisionary collegiate bodies made up of two or four citizens and a legal judge (BRASIL, 1965). After the election, political parties and other bodies can solicit from the Electoral Justice system copies of reports and archives generated by systems used in elections (TSE, 2017c). Weber (2018), a Justice of the Supreme Electoral Court, classifies Brazilian electronic ballot boxes as "totally reliable", because they can be audited.
Finally, the "validation, precision and test" dimension prescribes that institutions should validate, test and document the models and methods used in these systems to avoid discriminatory damage (ACM, 2017). Sources of errors and uncertainties, in the algorithms and data sources, should be identified and stored in order to help in any procedure to mitigate damage (DIAKOPOULOS, FRIEDLER, ARENAS et al., 2016). In addition, the systems and algorithms that guide decisions should be periodically revised in order to verify whether they continue to fulfill the intended objectives (STATS NZ, 2018).
In this last dimension, it has been identified that the Supreme Electoral Court performs the Public Security Test (TPS), which brings together specialists "[...] who try to corrupt ballot boxes and their internal and external components with the objective of discovering vulnerabilities of the system in relation to any possible violation of their results and breaking of voting secrecy" (BRASIL, 2016c, p. 26).
On the eve of the election, the Supreme Electoral Court performs an audit of the systems through a so-called parallel vote, when certain ballot boxes are selected at random to verify their compliance through voting by fictitious voters (TSE, 2017a). After the introduction of biometric identification in elections, the specialist Rezende (2018) argues that the simulated parallel vote has lost its meaning, given that any "cheating program", irregularly installed in a ballot box, could identify that it is being executed in a ballot box with a large percentage of biometric identification errors, frustrating the surveillance.
Digo Aranha (PAYÃO, 2017) points out that the improvement of the Brazilian voting system passes through not only the voting software security and its auditing processes, but also the implantation of mechanisms, such as the printed ballot, that allow the voter to verify whether the system registered his or her intentions correctly.

CONCLUSION
The field of algorithmic transparency studies, especially those linked to the public interest, has dealt with the importance of submitting to public scrutiny the "ingredients" and the steps taken in the preparation of the "recipe". It is not enough for digital services to show gains in efficiency and greater accessibility; they need to be able to attest to their legitimacy.
The effect of this has been that the demand for transparency has knocked on the door of public services provided by digital means. This is because there is no doubt that transparency is an indispensable condition for the promotion of accountability (GOEDE and NEUWIRTH, 2014). However, studies that identify the criteria to be followed by public algorithms that seek to be transparent are still incipient.
The transparency and accountability of digital public services gain greater relevance when they involve new technologies and democracy (KROLL, 2015), given the wide adoption of electronic voting systems in various countries (STOICA and GHILIC-MICU, 2016).
Our study of the Brazilian electronic voting system in light of the proposed analysis model has demonstrated its characteristics of transparency and accountability, making it possible to identify its strengths and weaknesses. One of the main findings points to the contrast to the high degree of adherence to the model in the consciousness dimension (100%) and the low degree of adherence in the accountability dimension (25%). The effect of this is the difficulty that citizens have in understanding and monitoring the current utilization of the electronic voting system which has resulted in distrust in the fairness of its operations, which leads to a need for constant publication initiatives about the system's functioning sponsored by the Electoral Justice system. A poll conducted in July 2018 by the digital company Avast found that there is little trust in the electronic ballot box system, given that 91.84% of Brazilian voters believe that the system can be violated (AVAST, 2018).
While some believe that the high level of distrust of the population in terms of the electronic voting system is due to a low level of education (JANINO, 2017), others believe there is a need to adopt mechanisms that allow voters to verify whether the system has registered their votes properly (PAYÃO, 2017), opening the possibility of an evaluation of state activities (GRIMMELIKHUIJSEN and WELCH, 2012).
Among the evaluated dimensions, the degree of adherence to auditing (66%) stands out, which permits data recovery in cases of suspected damage, which is indispensable for transparency and is important to how people view the system in general (FREITAS and MACADAR, 2017).
Transparency cannot be measured just by the mere publication of data, because it is not an end in itself, since it should fulfill a certain purpose (KLEIN, KLEIN and LUCIANO, 2018), which, in this case, is related to the need of the electoral process to promote broad participation, transparency and present the accounts of those involved and their democratic responsibility (PRADO, 2009). In the case of the electronic voting system, it has focused its accountability on the perspective of presenting governmental accounts and the responsibility of public agents to strengthen trust in the electoral process.
In 2018, the general elections held in Brazil counted on the participation of the Electoral Observation Mission of the Organization of American States (EOM/OAS), which noted the professionalism and the technical expertise with which the electoral process was organized, providing citizens with official information in a rapid manner and contributing to the certainty of the process (CHINCHILLA, 2018a). The mission recognized the efficiency, security and transparency of the Brazilian electronic voting system, and made recommendations for improvements (CHINCHILLA, 2018b).
Among the main recommendation of the mission were enlarging the size of the sample used in the parallel voting, increasing the presence of technicians from the political parties during the monitoring of the ballot boxes, and the creation of spaces for dialogue in which authorities and party representatives can work together to develop new measures which can increase the confidence of all actors in the system (CHINCHILLA, 2018b). The creation of these spaces for dialogue will help address the great need for participation to promote accountability (AKUTSU and PINHO, 2002;RUEDIGER, 2003).
Considering the growing importance of the transparency of governmental actions, the objective of this study has been to identify the mechanisms that promote the transparency and accountability of the Brazilian electronic voting system. The results obtained and the procedures adopted are relevant to the practices of public managers who desire to evaluate the transparency and quality of digital public services. The relevance of this empirical work is based on governmental transparency associated with accountability, understood as the problem of constructing and improving democratic institutions (LOUREIRO, TEIXEIRA and PRADO, 2008). The dimensions and items of analysis utilized make it possible to examine the transparency of algorithms and digital public services, which makes it possible to reapply them in the future and compare them with other digital systems and services. The theoretical contribution can be credited to an unprecedented combination of criteria designed to study the transparency of digital public services, vital for democracy, from the point of view of promoting accountability.
Future research can evaluate, in a longitudinal manner, the evolution of security and transparency characteristics of the Brazilian electronic voting system since its origin in 1996. A comparative analysis of the Brazilian system with systems used in elections in other countries would also be of interest. Finally, there is also the possibility of expanding the case analysis selected by using a triangulation of methods utilizing focus groups and Delphi techniques.