What is the quality of the regulatory impact analysis prepared by Brazilian regulatory agencies?

The regulatory impact analysis - RIA is a tool that seeks to identify the regulatory policy’s relevant impacts to support decision-making. Its use has become mandatory in Brazil, but no studies have investigated whether the quality of the analysis has reached the desired level in the country. Thus, the objective of this research was to evaluate the quality of RIAs carried out by Brazilian federal regulatory agencies. An instrument was developed and applied with 50 criteria for quality assessment in a set of 21 RIAs. The results indicated the good quality of the RIAs in some aspects, such as the definition of problems and objectives, but they pointed out weaknesses in others, such as the analysis of impacts and the comparison of alternatives. As an agenda for future studies, investigations of hypotheses and factors that may explain the absence of a quality standard for RIA in the country were proposed. e aplicado um instrumento com 50 critérios para a verificação da qualidade de um conjunto de 21 AIRs. Os resultados indicaram boa qualidade das AIRs em alguns aspectos, como na definição de problemas e objetivos, mas apontaram fragilidades em outros, como na análise de impactos e na comparação de alternativas. Como agenda para pesquisas futuras, foi proposta investigação de hipóteses e fatores que podem explicar a ausência de um padrão de qualidade da AIR no país.


INTRODUCTION
Regulatory Impact Analysis (RIA) has no single concept, and its use has varied depending on the country or context of application (Peci, 2011;Radaelli, 2010). In general, it is an analysis that seeks to identify the relevant impacts of a regulatory policy, in order to support decision making and achieve balanced and effective policies (Ellig, McLaughlin, & Morrall, 2013;Jacobs, 2004).
RIA has been widely spread around the world, with growing acceptance (Adelle et al., 2016). The adoption of RIA in regulatory processes started slowly in the 1970s, when it was adopted in the United States, and advanced a lot in the 1990s and 2000s, when it reached most countries of the European Union and member countries of the Organization for Economic Cooperation and Development (OECD) (Francesco, 2012). Recently, it has also progressed in developing countries as a strategy to improve governance (Carvalho, Costa, Marques, & Cordeiro, 2019).
In Brazil, the government showed a first institutional commitment to RIA in 2007, by creating the Program for Strengthening Institutional Capacity for Regulation Management (Proreg). Its goal was to adopt measures for regulatory improvement, which included encouraging the use of RIA in policy making in the country (Peci, 2011). Despite the efforts, RIA had a modest progress until mid-2018. Its application was incomplete, non-systemic, and not understood as a government strategy (Castro, 2014).
In recent years, the Brazilian government has fostered the topic of RIA, and passed two laws 1 and a presidential decree 2 that made its use mandatory in regulatory policy formulation. Since April 15, 2021, Brazilian regulatory agencies -at the federal level -started to use RIA as a requirement for policy making Despite the growing use of RIA around the world, there is still a shortage of studies indicating if the quality of the analysis has achieved the intended analytical depth. The available papers are concentrated in the United States and Europe, and make only partial assessments on the diverse aspects of quality (Achtnicht, Rennings, & Hertin, 2009;Belfield, Bowden, & Rodriguez, 2019;Fritsch, Radaelli, Schrefler, & Renda, 2013;Saab & Silva, 2021b). The results of these studies indicate that, in recent decades, RIA quality was low, and to identify ways to improve it involves carrying out new evaluations in different contexts (Belfield et al., 2019;Ellig & McLaughlin, 2012;Hahn, Burnett, Chan, Mader, & Moyle, 2000;Saab & Silva, 2021b;Souto-Otero, 2013).
Based on the available knowledge on the subject, there is a gap that justifies the present study, whose objective was to evaluate the quality of RIAs done by Brazilian federal regulatory agencies. Hence, this paper addressed a reality not yet explored in previous studies -Brazil'sand used more comprehensive evaluation criteria, which can contribute to a broader debate on RIA's quality.

ASSESSMENT OF RIA QUALITY
The use of knowledge, evidence, and information to support decision makers is increasingly frequent in developed democracies (Staroňová, 2014). When a decision maker seeks knowledge before choosing a policy, he/she aims to know and measure the expected impacts, in order to make better decisions and track its results (Desmarais & Hird, 2014).
RIA can be considered a tool that allows approaching science and knowledge on regulatory policy making. Its use can mean an important strategy to use qualified information and arguments for policy decisions (Costa, 2016).
RIA defines the problem and intervention's goals, and compares the different alternatives available to the regulator (Jacobs, 2004;Radaelli, 2004Radaelli, , 2010. Throughout the process, it uses methods and evidence to support each step, as well as encourage wide social participation. In the final part of the analysis, it presents the most favorable regulatory path for society (Radaelli, 2010).
RIA can also play a relevant role against regulatory capture. The use of standardized methods to assess costs and benefits, and the availability of results for public scrutiny can help detect situations where data are manipulated in order to promote regulations that meet the interests of certain groups (Livermore & Revesz, 2013;Saab & Silva, 2021a).
Merely carrying out RIA does not ensure good regulatory policies. It is necessary to overcome misgivings about its adoption, develop technical capabilities, and secure greater political commitment to its use (Carroll, 2010). Impact analyses should have appropriate quality in order to provide good inputs to the decision maker (Daskal, Ayalon, & Shechter, 2019;Ellig et al., 2013;Fritsch et al., 2013).
The next subsections present dimensions, criteria, and approaches that have been used to assess RIA quality, and provide measures for improvement.

Dimensions and evaluation criteria
In general, RIA quality assessment can be done by investigating four dimensions ( Figure 1): i) RIA's formal aspects; ii) the use of social participation; iii) the use of evidence in the process; and iv) its usefulness for decision-making (Saab & Silva, 2021b).  In the first dimension, we should assess if the formal aspects of RIA were properly conducted and registered. The aim is to investigate criteria that measure the appropriate definition of the problem and of the regulation goals (Staroňová, Pavel, & Krapež, 2007), in addition to the identification of regulatory alternatives (Dudley et al., 2017), as well as their comparison (Souto-Otero, 2013), in order to recommend the best regulatory option to the decision maker (Ellig & McLaughlin, 2012).
In the social participation dimension, we should assess criteria that indicate if RIA's development process was transparent and participatory (Ellig & McLaughlin, 2012). It is important to investigate if the government authority involved society, encouraged data collection, captured perceptions from different social groups, and answered their needs (Staroňová, 2010;Staroňová et al., 2007).
As for the use of evidence in the process, we should look for criteria that indicate if the arguments and assumptions used in RIA were based on properly proven and verifiable elements, facts, data, and information (Russel & Turnpenny, 2009;Souto-Otero, 2013).
Finally, the dimension of usefulness for decision-making is peculiar. It is not an intrinsic and verifiable element in the RIA report, but we know that there is a bidirectional correlation between the quality of the analysis and its use in decision-making. When RIA has a better quality, it influences the decision-making process more, just as the decision maker's intention to use it tends to increase its quality (Ellig & McLaughlin, 2012).

Approaches for assessing RIA quality
The quality of RIA is measured by applying different approaches (or methods) to the pre-established criteria and dimensions. In the last 40 years, we found four approaches in the literature: i) checklist; ii) point scale; iii) retrospective comparison; and iv) case study.
The checklist consists in verifying the criteria and contents that RIA should have, by assigning "yes" or "no" values. The point scale is similar to the checklist, but sophisticates the assessment by assigning points to each criterion. The case study involves a qualitative investigation on RIA, based on capturing and analyzing perceptions and information. Finally, a retrospective comparison assigns quality to RIA, according to the accuracy in estimating costs and benefits (Saab & Silva, 2021b). Hahn et al. (2000) were pioneers in proposing studies to assess RIA quality. According to the authors, RIA was created as an instrument to facilitate the development of efficient and effective regulatory policies, but the level of information provided by the analyses was insufficient to achieve that goal. Assessing RIA quality would be an essential initiative for its strengthening and, consequently, for improving regulatory policies.

Related studies
Some studies mark the trajectory of knowledge building on the subject. To assess the quality of 45 RIAs conducted in the United States in 2008, Ellig and McLaughlin (2012) proposed 12 criteria and used the point scale approach. The criteria generally measured RIA's ability to define the problem, identify and compare alternatives, and estimate the costs and benefits of regulation. In addition, they considered the ease for a citizen to locate and understand the RIA report, and if RIA affected the decision. RIAs' examination showed that the average quality of the analyses was low, with many weaknesses related to problem definition and use of evidence for decision-making. Fritsch et al. (2013) used the checklist approach to evaluate 773 RIAs from the United Kingdom and European Union, between 2005 and 2010. They used 18 assessment criteria, focusing on those related to cost and benefit calculation, impacts, quantification, and profitability -"we left aspects such as problem definition, consultation, monitoring, and evaluation for further work arising from this project" (Fritsch et al., 2013, p. 447). The result showed that cost and benefit analysis, quantification, and monetization were problematic issues in RIAs.
More recently, Belfield et al. (2019) assessed the quality of 28 RIAs that supported regulatory policies in the education area, in the United States. The authors used 11 criteria and the checklist approach. Similar to Fritsch et al. (2013), their examination of RIA quality emphasized criteria linked to the determination of costs and benefits. Other aspects of the RIA process, such as problem analysis, goals, and social participation, were not used for assessing quality. The study concluded that, in the few situations where RIA was used in educational policies, methodological standards were low, costs were underestimated, and there was little transparency regarding methods and assumptions. In addition, benefits were rarely estimated, and the adopted method did not ensure high reliability levels.
Some studies have contributed to evaluating RIA quality in different contexts, helping to understand its strengths and weaknesses. Improving assessment methods, approaches, and criteria, as well as conducting new evaluations, can help identify ways to improve the analysis. The next section presents the methodological procedures adopted to assess the quality of RIA in Brazilian regulatory agencies.

METHODOLOGICAL PROCEDURES
According to Yin (1994), an empirical investigation is based on a research plan that relates data and objectives to reach conclusions. Figure 2 presents the steps and procedures used in this study, between its inception and conclusion. In the first stage, we explored the scientific literature on RIA quality, in publications found on Google Scholar, Web of Science, and Scopus databases. We also examined official documents (laws, decrees, and guides) that regulate the topic in Brazil. With these scientific studies and official documents, we designed a preliminary instrument to assess RIA quality in Brazil. The preliminary version of the instrument was sent to a referee -a PhD in Social Sciences with extensive experience in public management, regulation and RIA -who judged its adequacy. The final version was prepared after adjustments resulting from referee's observations.

FIGURE 2 RESEARCH PLAN
The final version of the assessment instrument had 50 quality criteria, and was prepared for applying the point scale approach. The proposed methodologies cover the dimensions of quality investigation suggested by Saab and Silva (2021b), and enable more complete assessments than those available in the literature. We chose the point scale because this approach differs from others, by allowing a qualitative perception of how well each quality criterion was evaluated (Saab & Silva, 2021b). To apply the instrument, we adapted a point scale from Ellig and McLaughlin (2012), as shown in Box 1. In the second stage of the study, we conducted a survey of all RIA reports that followed regulatory policies approved between April 15 and December 31, 2021. We requested information from the 11 federal regulatory agencies, through the Brazilian government platform 'Fala.BR' , used to handle requests for information access (Bourges, 2020). The choice of period aimed to select only RIAs prepared after its mandatory implementation in Brazil, when concepts and practices were defined. All regulatory agencies answered the request for information through the 'Fala.BR' platform.
After the RIA survey, we carried out a floating reading of the reports, in order to check if the documents fit the research scope. Next, we organized them, and applied the quality assessment instrument to each report. The two authors of this paper conducted the evaluation. The first analyzed the content of the reports and applied the evaluation instrument; the second reviewed and discussed the results of the analysis.
In the third and last stage, we analyzed and described the results achieved through the assessment of RIA quality.

RESULT ANALYSIS AND DISCUSSION
We carried out the analysis and discussion of results by assessing the quality of a set of 21 RIAs prepared by Brazilian federal regulatory agencies. Among the 11 agencies, only Anvisa, Ancine, and ANM did not have their RIAs assessed, because they did not approve regulatory policies, in the period from April 15 to December 31, 2021, following the new concepts and practices for RIAs defined in the Brazilian legislation, as shown in Box 2. After applying the assessment instrument, we present the results in seven subsections, explained below: (i) problem analysis and definition; (ii) goal setting; (iii) identification of regulatory alternatives; (iv) analysis of alternatives' impacts; (v) comparing alternatives; (vi) indication of the best alternative; and (vii) transparency and social participation.

i) Problem analysis and definition
Proper problem analysis and definition is the most important stage in designing a regulatory policy (Nweke, 2011). When analyzing a problem, the undesired situation must be deeply understood for the intervention to be effective, justified, and targeted to real and legitimate problems of society (Souto-Otero, 2013).
The results of the evaluation showed that the analysis and definition of the problem were not neglected by Brazilian regulatory agencies but were stronger in some aspects than in others (Table 1). In all RIA reports evaluated, there was a section dedicated to understanding the undesired situation. In general, there were significant efforts to delimit and define the problems to act upon. However, some regulators did not understand the problem correctly, as "the lack of regulation" or "the need for regulation". In these cases, they confused the problem with a possible solution -the regulation. Some also presented long descriptions of the problem, without clearly delimiting it.
In general, regulators showed no familiarity with the concept 'problem nature' . Few RIAs showed some level of discussion on market failures or regulatory failures. Best practices recommend identifying these failures, so that agencies understand the problem, the need to act upon it, and promote effective regulation in economic and welfare terms (Dudley et al., 2017;Ellig & McLaughlin, 2012).
Another issue that requires improvement by Brazilian agencies is the use of evidence on the identified problem. The research indicated that the use of evidence was incipient in the RIAs assessed. In general, regulators mentioned their own understanding of the undesired situation they intended to face (problem), and used little data and evidence to support their points of view. Not even the search for data and information on international experiences was used systematically. Given the low body of evidence in RIAs, it would be appropriate to carry out an uncertainty analysis on the problems (Ellig & McLaughlin, 2012), but none of them addressed uncertainties objectively.
Problem analysis and definition, when properly done, include the identification of its causes and consequences. Often, policies are based on the analysis of the factors that cause the problem and the resulting consequences. The development of a diagram or problem tree can be crucial in this process (Mota et al., 2020;Snowdon et al., 2008).
In Brazilian RIAs, regulators rarely used problem diagrams to illustrate the causes, root causes, and consequences of the problem they intended to address. Most of the time, causes and consequences were little explored. Analyses of trends in problem evolution, essential to guide the urgency and need for regulators' action, were also rare and fragile.
Despite the weakness of the analysis on problem's consequences, Brazilian regulators made good mappings of the actors who suffered the effects of the problem. The identification of the affected players was present in all investigated RIAs. In the best analyses, regulators recognized different stakeholders, and discussed how each of them was affected. In lower quality evaluations, regulators just listed the groups of players, using generic names that did not allow their exact identification. They used terms like 'users' , 'agency areas involved with the topic' , and 'investors' .
Finally, the last criterion related to problem analysis and definition refers to the legal basis. This is about defining the legal mechanisms that support government action upon an identified problem. In general, this criterion was very well evaluated in Brazilian RIAs. As a rule, there was citation and explanation of the laws, regulations, and instruments related to the institutional competence for acting on the problems. The good quality of RIA in this aspect prevents regulators from wasting time and effort to solve problems upon which they have no legal competence to act.

ii) Goal setting
After identifying the problem to be faced, the regulator must define the intervention goals. This stage can be the change from the negative statement of the problem into a positive statement of goals (Snowdon et al., 2008), which need to be clear and measurable in order to guide policy formulation (Souto-Otero, 2013).
Goal setting was found in almost all of the 21 Brazilian RIAs evaluated. In just one, the intervention's objective was not presented. In general, goals were clearly described and connected to the problem. This logic, followed by Brazilian regulators, is important to ensure that goal achievement addresses the confrontation, mitigation, or elimination of the problem.
Although the definition of goals has generally fulfilled its role in the assessed RIAs, the score presented in Table 2 shows that the quality of the information provided could be better. Especially when the problem was not well defined, the regulator used generic and non-measurable verbs to determine the intervention goals. In some RIAs, they were established based on expressions like "improve consistency", "keep a favorable regulatory environment", or "optimize efforts". These are ambiguous objectives that allow several interpretations, which can lead to developing policies that are ineffective, detached from the problems, and difficult to monitor results.

iii) Identification of regulatory alternatives
When defining the goals of an intervention, the trend is to consider only the regulations for achieving them (Hepburn, 2006). However, the same objective can be achieved through several paths. In RIA, regulators must identify all feasible alternatives (Souto-Otero, 2013).
In this process, besides thinking of different possibilities for regulation, regulators must think on non-normative alternatives -regulatory means that are more flexible and less prescriptive than traditional regulation (Hepburn, 2006). Regulators should also consider the possibility of "no action". An intervention should only be taken if it is better than the alternative of doing nothing (Mazzocchi, Ragona, & Zanoli, 2013).
In all RIAs assessed in this study, Brazilian agencies presented regulatory alternatives to achieve the intended goals. As indicated by the scores in Table 3, the use of the "no action" alternative was more consistent than the use of non-normative alternatives. The main limitation to the use of the "no action" option was its early elimination. Some regulators excluded this possibility based on superficial arguments. In the case of the non-normative alternatives, the main weakness was their limited use in the investigated RIAs. On the rare occasions when regulators suggested non-normative alternatives, they were eliminated early or considered only as additional measures to traditional regulation. We also noticed that the language used by Brazilian regulators in RIA reports was oriented towards traditional regulation. In the texts, we identified little willingness to adopt more flexible regulatory alternatives. The preference for known and traditional norms was already found in previous studies that evaluated RIA quality in other countries (Posner, 2003;Radchenko & Parshina, 2014;Russel & Turnpenny, 2009). According to Hepburn (2006), this preference may occur because of regulators' perception of risk in the process. They are not aware of many successful experiences with regulations other than traditional, have little information on the use of these alternatives, and are afraid of the government losing control over the problem, if it does not adopt conventional regulatory measures.
To deal with regulators' risk aversion, good practice in RIA suggests the use of risk analysis tools for the alternatives. By gathering information on risk, they could overcome their constraints and concerns (Hepburn, 2006). Despite its relevance, the risk analysis of alternatives was seldom done in the assessed RIAs. There was a huge difficulty of regulators to consider regulatory alternatives in the process, which hindered the quality of Brazilian agencies' analyses.

iv) Analysis of alternatives' impacts
If implemented, each regulatory alternative will have different impacts on society. RIA should include a strong analysis to provide a complete scenario of the harmful and beneficial effects of each alternative. It should explain how each alternative will contribute to achieve the regulatory goals (benefits) and the costs incurred to meet the established requirements. It is essential to present evidence, in order to support the analysis (Dudley et al., 2017).
In previous studies on RIA quality assessment, authors found that social, environmental, and economic impacts, and those for complying with regulation, were the most important (Fritsch et al., 2013;Russel & Turnpenny, 2009;Staroňová, 2010). In Brazil, government guidelines further suggest that impacts on micro and small firms, competition and competitiveness, international trade, security, and public budget be included in the analysis (Ministério da Economia, 2021).
As shown in Table 4, RIAs prepared by Brazilian regulatory agencies were weak in covering these impacts. There were few analyses that considered the effects on the economy, health, and environment. The effects on competition and competitiveness, security, micro and small firms, international trade and public budget, although explicit in government's guidelines, were not taken into consideration properly.
In general, RIAs conducted in Brazil have considered two types of impacts. Regulatory coststhose arising from compliance with obligations imposed by regulation -and other impacts. These include more specific effects, such as "customer service", "participation in concession's management" and "conciliation of interests". Another weakness found in RIAs was the rare use of evidence in the analysis of these impacts. Regulators, in general, used just impact expectations, developed through simplistic arguments, with little support from evidence.

v) Comparing alternatives
The comparison of alternatives, based on calculated impacts, should follow consistent methods that assess all positive and negative effects relevant for decision-making (Jacobs, 2004). It is possible to compare them through the appropriate use of quantitative methods of analysis, such as cost-benefit, multicriteria, and cost-effectiveness (Aquila et al., 2019). Qualitative methods of comparison can also be used, as long as procedures for maximizing the consistency of the analyses are adopted (Jacobs, 2004).
A key element to guide the comparison of alternatives is to present the "no action" alternative as a baseline. This means that the differences between the baseline and the incremental changes imposed by each alternative will indicate the best regulatory alternatives available (Dudley et al., 2017).
As Table 5 shows, the use of the baseline was a neglected practice in the investigated RIAs. Moreover, in only a few cases they used consistent comparative methods. Justification for choosing the methods was also not presented systematically. As a rule, Brazilian regulators carried out the comparison of alternatives based on tables or texts that indicated their own understanding on the advantages and disadvantages of each alternative. These procedures have been classified by many regulators as a 'qualitative method' for comparing alternatives. These comparisons were often done without any methodological accuracy, and mistakenly classified as 'qualitative' . The risk of this methodological flexibility resulting in analyses without method had already been mentioned by Jacobs (2004) and Lussis (2004). If Brazilian agencies wish to use qualitative methods to compare alternatives, they should adopt procedures for data collection and analysis to achieve consistent results.
Regarding the use of quantitative methods for comparing alternatives, the practice proved to be incipient. The most used was multicriteria analysis, but without a methodological standard. Some regulators applied multicriteria analysis with accuracy, while others ranked very simple analyses as multicriteria. Because of the rare use of quantitative methods, Table 5 shows that the quality scores of RIAs related to quantification criteria and the use of discount rate and sensitivity analysis were very low. Reports indicated that the agencies did not have the necessary maturity to use quantitative methods consistently.

vi) Indication of the best alternative
The indication of the best alternative is the direct result of the impact analysis and the comparison between different alternatives. An alternative is recommended to the decision maker when it is identified as better than the others (Ellig & McLaughlin, 2012). The recommendation of the best regulatory alternative was present in all RIA reports evaluated. In general, Brazilian regulators concluded the rationale for comparing alternatives by highlighting the one that proved to be the best for society. However, in some cases, the arguments supporting the choice of the best alternative lacked the expected consistency and clarity. Incompleteness in the stages of impact analysis and comparison of alternatives led to unsubstantiated recommendations.
In addition, Table 6 shows that Brazilian regulators provided limited information on the expected risks and effects of adopting the best alternative, and showed poor information on its implementation, monitoring, and evaluation.

vii) Transparency and social participation
Throughout all RIA stages, regulators may find gaps in knowledge and information for developing sound and thorough analyses. Therefore, a quality RIA needs to adopt systematic procedures for data collection and analysis, in order to provide realistic and useful results to the decision maker (Kirkpatrick, Parker, & Zhang, 2004).
Moreover, it is up to the regulator to encourage society to critically comment on RIA (Ballantine & Devonald, 2006). When social participation is effective, and different standpoints are considered in the analysis, we achieve greater credibility and legitimacy, and the chances of the regulatory policy attaining its goals increase (Radaelli, 2004). For society to participate, it is essential that the RIA process be transparent. The reports and documents related to the analysis should be public and prepared in a simple and accessible language (Saab & Silva, 2021a).
As shown in Table 7, transparency proved to be a well evaluated element in the investigated RIAs. The scores regarding the access to reports and the ease of reading the information were high. Most reports were found on the regulatory agencies' websites, and the language adopted was appropriate for society's understanding. In this aspect, we identified an opportunity for improvement, by better structuring the executive summaries. Sometimes they were used to introduce the report's content, and not to summarize its main findings. As for society participation, we identified some elements that need improvement. In some cases, regulators encouraged the participation of society after completing RIA, through public consultations on draft regulations. However, this was a late moment. When participation occurs when there is already a government positioning, the likelihood that other standpoints will be considered in the regulation is lower (Saab, Bermejo, Garcia, Pereira, & Silva, 2018).
Furthermore, cases where participation was restricted to areas and departments of the agency itself were not uncommon. There was no systematic concern for ensuring that groups affected by the problem were heard. Even when regulators received contributions from society, they seldom showed the analyses and justifications for accepting or rejecting them.

FINAL REMARKS
This study advanced previous research and assessed the quality of a set of 21 RIAs prepared by Brazilian federal regulatory agencies, by developing and applying an evaluation tool with 50 quality criteria.
The results of the evaluation showed that RIAs developed in Brazil were appropriate in some aspects, and weak in others. The definition of the regulatory problem and the objectives of the intervention showed good quality, as well as the process transparency, which also received good scores. On the other hand, the results indicated important weaknesses, especially in investigating impacts and comparing regulatory alternatives. The impacts considered in the analysis were limited, and the methods used for comparing alternatives were poorly consistent.
We raised some hypotheses to explain the absence of an analytical standard in RIAs done in Brazil. Insufficient technical capacity to carry out sophisticated impact analyses can explain the methodological limitations and weaknesses found. In addition, interest groups may have influenced RIA quality, through pressure and resistance to its use in certain contexts. Likewise, political commitment to conducting RIA may have varied according to the interests involved in the regulatory proposals under analysis. Finally, the recent institutionalization of RIA in the country may have generated suspicions and doubts regarding its use, which must be overcome in order to achieve the desired standard of quality in the analyses.
Future studies that investigate in depth the hypotheses raised can contribute for building knowledge on the subject. Interviews with actors involved in the implementation and execution of RIAs in Brazil can provide insights and new data for explaining RIA quality in the country.
The results of this research can contribute for managers and civil servants to know the aspects of highest and lowest quality in RIA practice in the country. Resources and efforts can be oriented to train and qualify regulators more effectively. Additionally, the evaluation instrument proposed in this research can be adopted by public administration bodies to assess quality and improve impact analyses.
From a theoretical standpoint, this research added knowledge to previous studies that evaluated the quality of RIA around the world, and confirmed the need for improvements in the analyses. In addition, we designed a new evaluation tool, based on the literature, which can be criticized and improved, so that other assessments can be carried out in different contexts and realities.