Results-based contracs: weaknesses in transparency and low accountability

Resumo Esta pesquisa analisou a transparência e a responsabilização na contratualização de resultados entre o Governo do Estado de São Paulo e as organizações sociais (OS) de cultura, com base nos julgamentos do Tribunal de Contas do Estado de São Paulo (TCE-SP). Adotou-se a perspectiva instrumental da transparência, para analisar os inputs e os outputs do processo de contratualização. Os resultados apontam fragilidades da transparência, especialmente nos aspectos de mensurabilidade e tempestividade da informação, afetando negativamente a responsabilização da OS e do governo estadual diante da população. Indica-se, também, a necessidade do governo aprimorar os mecanismos de controle interno na análise dos contratos.


INTRODUCTION
This research analyzed the implementation of transparency and accountability in the Brazilian public sector by studying the contractualization of results between the government and social organizations (SO) in the state of São Paulo, Brazil. The goal was to advance knowledge on transparency and governmental accountability by studying the monitoring the management contracts of SO and based on the conceptual discussion about the concepts of transparency and accountability.
These concepts are usually treated as similar (Zuccolotto & Teixeira, 2019;Zuccolotto, Teixeira, & Riccio, 2015), leading to the understanding that a greater amount of information available to the public automatically makes the agents more accountable (Fox, 2007). This correspondence is not very effective, especially in experiences such as the contractualization of results between government and private entities (Head, 2006;Rocha, 2011). This type of experience depends more on reliable and timely information, which is agreed upon in the initial phase of the contract and crucial for the visualization of the results during account-giving.
Studies on transparency usually aim to measure the volume of information made available on the internet and do not link such amount to the accountability for the results (Zuccolotto, Teixeira, & Riccio, 2015), also failing to consider transparency as part of the evaluation process of public policies (O'Neill, 2006), differentiating, for example, input, process and output information in contracts with private partners (Reynaers & Grimmelikhuijsen, 2015). Currently, there are no studies on transparency and accountability based on the contractualization documents, with the literature only analyzing the information from websites of public agencies or transparency indices. The "positive" aspect of transparency is emphasized to the detriment of its functional aspect due to the dominant concern with combating corruption (Heald, 2006) Transparency and accountability are common terms in discussions about State reform; however, the Brazilian experience faced the spreading of contractualization as the alternative for the high demand for services and efficiency and for fiscal control (Costin, 2005;Duarte, 2010;Alcoforado, 2009). This process was unrelated to State reform, occurring in a fragmented way. Rocha (2011) draws attention to the importance of the context of a given public policy to achieve accountability: the accountability of agents is not based solely on the laws but mostly on performance standards and expected results, given that the accountability model emerging from the New Public Management does not foresee or explain the mechanisms by which such agents will be held accountable in relation to society.
Such problem can be observed in the experience with SO, especially in São Paulo, the State with the most consolidated experience in the contractualization of SO in Brazil, especially in the areas of health and culture. The contractualization of cultural activities began in 2004 and has grown since then, being based on management contracts of goals and results as the guiding instrument.
All cultural equipment in São Paulo are managed by SO (ACAM Portinari, 2014) under the monitoring of the Secretaria da Cultura do Estado (SEC -State Department for Culture). According to Portal da Transparência of the Stete of São Paulo, contracts with SO of culture currently reach 2.2 billion, to be spent until 2023. This reinforces the need for its alignment with the strategic planning of the policies of the area, and the concern with the effectiveness of the results for society.
Having the understanding of the relationship between transparency and accountability in the context of decentralized service delivery -which presupposes the integration between information and result -as our theoretical challenge, we find that organizations are an interesting object of analysis: demanding accountability is crucial in this model due to the need for accountability of those who manage public resources (Porta, Santos, & Palm, 2015). This model was implemented in a fragmented way, aimed at solving fiscal issues to the detriment of improvements in performance. Moreover, the literature has pointed to a lack of transparency in the choice of entities, or even in the disaccreditation of others (Matta, 2013;Romão Netto, 2015). However, there is no analysis of how the process of contractualization of results affects the accountability of agents and especially how model transparency interferes with this accountability.
This study thus fills a theoretical and empirical gap: we contrast the presumption of literature that the greater amount of information makes organizations work better (Heald, 2006;Fox, 2007). The empirical gap is the lack of analyses on how information about the process of contractualization of results promote accountability of the agents involved. Such analytical perspective is fundamental to understand the effectiveness of transparency and its relationship with accountability.
We assume that transparency influences accountability when it constitutes a reliable information system that can show the performance of agents and their results. Furthermore, we assume that the structure of transparency mechanisms shapes their impact on public policies -especially in the accountability of agents -, being crucial for a study that analyzes the role of transparency in accountability.
The subject of analysis are the final opinions of Tribunal de Contas do Estado de São Paulo (TCE-SP -São Paulo's State Court of Auditors) on two instruments for accountability of the results of SO of culture: management contracts (MC) and account-giving (AG). The requirements for documentation and the type of analysis of the TCE-SP on these instruments allows an analysis of transparency and its effect on accountability since they present all phases of the management contract and the responsibility of the actors. The analysis of the opinions of TCE-SP was the instrument to verify whether the flaws related to transparency exerted influence on the trial of an irregularity, and whether they interfered in accountability.

TRANSPARENCY, ACCOUNTABILITY, AND SO OF CULTURE IN SÃO PAULO
The current literature on transparency is no longer focused on the debate about the active dissemination of information, but on the insufficiency of the availability of large volumes of information to ensure accountability, and on questioning the rhetoric that transparency automatically implies in being accountable (Fox, 2007;O'Neill, 2006;Heald, 2003;Zuccolotto & Teixeira, 2019). This literature understands that transparency has instrumental function, i.e., it is not an end in itself but part of the process of evaluation of a given public policy, being an important requirement but not sufficient for accountability (Fox, 2007). Hence the importance of separately analyzing transparency and accountability.
Accountability is understood as holding the agents of a management contract responsible not only for the regularity in the application of public resources but, above all, for the achievement of results (Duarte, 2010). The concept of transparency has been developed linked to discussions on legitimacy (Baume & Papadopoulos 2012), especially in the fiscal area (Kopits & Craig, 1998). Transparency is currently analyzed as an institutional relationship through which the provision of detailed information on the performance of agents and on the achievement of goals is a requirement to monitor the performance (Meijer, 2012;Zuccolotto & Teixeira, 2019).
In the analysis of transparency, Heald (2006) differentiates between event transparency and process transparency: the first corresponds to the disclosure of inputs, outputs and outcomes of services -these represent points that are externally visible and measurable. Process transparency, on the other hand, corresponds to the organization's operational procedures, which are not measurable in the same way as events, although they can be described if the information is available. This type of transparency is identified in the contractualization of results with private entities, in which the focus is on achieving the results foreseen in the contract, with greater flexibility for internal processes. Pinheiro, Pinho, and Bruni (2018), studying accountability in hybrid organizations, reveal that their governance structure involves more actors and forums, which is positive for accountability. The democratic, constitutional and performance dimensions would be below expectations, substantiating a "symbolic accountability", calling attention to the analysis of the instruments and practices that promote accountability. For the authors, management, control, supervision and accountability are elements of governance, where the practices adopted by the first three promote accountability, and the integration of each one with a view to improving governance is fundamental.
The model of contractualization of services by the State is thus a privileged object for the analysis of transparency, since there is still uncertainty about the quality of information -financial parameters, level of service, accuracy of reporting and monitoring systems -at different stages of the contract with private parties (Pinheiro, Pinho, & Bruni, 2018). The flow of events is easily visualized in the contractualization of results with Social Organizations, whose focus is achieving the goals foreseen in the contract -which has a well-founded arrangement of controls that depend on the availability of information to achieve accountability.
The governmental control of Brazilian SO takes place in three stages: a) qualification; b) monitoring; and c) supervision (Alcoforado, 2010). The first two are under the responsibility of internal control, and the last of TCE-SP.
The State of São Paulo is the oldest entity with OS contracting experience in Brazil. It started in the health sector, with the contracting of public hospitals during the management of Mário Covas (1995)(1996)(1997)(1998)(1999)(2000)(2001) (Ferreira Junior, 2003). The contracting with OS of culture was instituted by the same normative act as health (State Complementary Law No. 846, 1998), but the first contracts in culture were signed in 2004 and have been expanding its scope since then (Souza & Almeida, 2010) In São Paulo, the entities that make up the internal control of the model are: a) Secretariat (monitors the execution); b) OS board of directors (approves institutional acts of the contract); c) Secretariat's contract management unit (monitors and directs contracts); and d) Evaluation Committee (with equal composition, supervises results and issues opinion that will subsidize Secretariat and TCE-SP).
The importance of analyzing SO of culture in São Paulo is due to some specificities and characteristics of this sector, which are briefly explained below.
The measurement of results in the area of culture is still problematic. The health area is more advanced since it presents nationally standardized indicators. The production indicators adopted in São Paulo for cultural institutions of distinct nature are too simplified and homogeneous (Pacheco, 2009). The difficulty in specifying indicators in culture is largely due to the intangibility of resultsespecially those that refer to the experience of the public in events, which is the main objective of the activities. There are other difficulties such as activities whose execution goes beyond the budget cycle -e.g., opera seasons, in the musical area -and extremely specialized activities for which there are very few suppliers thus making price comparison a difficult task -e.g., the museological area. Moreover, these specificities are hardly contemplated by the control mechanisms, which are homogeneous and, due to legislation, not susceptible to adaptations.
Finally, the funding of the cultural area in Brazil is very dependent on the public sector -whether via the budget, cultural promotion or in the form of tax incentives (Matta, 2013). This means that, for most citizens, the activities offered by the State are their only possible contact with art and culture, hence the importance of analyzing the cultural area -even if its percentage in budget expenditure is significantly lower than areas like education and health.
In the area of culture, the literature has pointed to failures of accountability due to the lack of transparency in the choice of contracted entities and the reasons for disqualification of OS (Matta, 2013;Romão Netto, 2015). Another weakness is the underestimation of goals, which define actions that already constitute routine organizations and do not imply efforts to increase service efficiency (Fiore, Porta, & Duarte, 2011;Freitas, 2010). Thus, the literature still indicates the importance of analyzing OS control, as well as the lack of studies on the performance of the main external control mechanism (in this case, the TCE-SP) in OS surveillance.

METHODOLOGY
The analysis of the relationship between transparency and accountability was made by studying the opinions of TCE-SP on transparency events (inputs and outputs of the contract) and its influence on the judgment of an irregularity. The opinions were identified, quantified and analyzed. Following, it was verified how these opinions were related to achieving the accountability proposed in the model of SO.
Based on the specificities and limitations of government transparency for achieving accountabilityspecifically the accountability of results -, this research listed topics for the evaluation of the transparency in the opinions of TCE-SP to verify how the requirements for information on the account-giving of SO of culture could be identified in terms of transparency events and how such information showed the results of the management contract and, consequently, provided, or not, the accountability.
The opinions of irregularities made by TCE are based on Consolidated Instructions no. 01/2008. For example: a) divergences in financial statements; b) lack of disclosure of the documents necessary for the agreement of the management contract; and c) not achieving the expected results.
The Court's judgment can have three results: a) "regular" (management acts considered to be legal), b) "irregular" (justifications about the failures are not accepted by the Court, which applies sanctions) or c) "regular with reservations" (the relevance of irregularities is insufficient to disapprove processes, with recommendations for adjustments).
The sample used consists of TCE-SP opinions with the results "regular with reservations" or "irregular" on the MC and AG materials of the SO of culture of São Paulo.
The period analyzed was between 2004 and 2015; 2004 was the year in which the first contracts were signed. Since data collection occurred in the second half of 2015, the period analyzed covers until June 2015.
The opinions were obtained through requests for information via the Electronic System of Information to the Citizen (e-SIC) of TCE-SP. For the period analyzed, there were 53 opinions, considering that 23 were related to MC and 30 to AG.
Using Heald's typology on transparency events, input documents were considered as the ones required for signing the contract (well-regulated and easily measurable); output documents are "deliveries" of the contract. Outcomes are not analyzed by the external monitoring, so they were not included in the study. Figure 1 presents a brief description of inputs and outputs. The opinions by TCU-SP were read in full, and from this quantified and separated by theme (management contracts and account-giving), and the events (inputs and outputs) and processes were classified. The recurrence of the events was related to the opinion of TCE-SP (regular with reservations or irregular), seeking to identify the cracks in the accountability of results via the qualified analysis of these opinions.
Although the management contracts and account-giving are processes analyzed separately by TCE-SP, they complement each other to enable the accountability of results: at first, in the evaluation of the transparency, economy and regularity in the choice of SO, and achieving the goals; and subsequently, in the implementation of the expected results and in the correct application of public resources.

RESULTS
The analysis was conducted considering the two themes analyzed by TCE-SP (management contracts and account-giving) and their peculiarities, and according to the phases of transparency events (inputs and outputs), the type of information to which they refer to, and their recurrence in the sample.
"Inputs" are management contracts of procedures and documents that make up the phase of choosing the SO and signing the contract. They are measurable by the public through publications in Diário Oficial do Estado de São Paulo (DOSP) and on the website of SEC.
The outputs are "deliveries" related to the results of the MC and are composed of the analysis of goals and by the identification of irregularities in the reports that should show the results. Unlike the theme "management contracts" in which inputs are predominant, most opinions in account-giving are directed to procedural aspects and not to results thus making it difficult to visualize the effects of transparency -or the absence of it -in achieving accountability in this phase of the process.
The following Box summarizes the specification and quantification of the analysis object: The lack of transparency in the inputs of the management contracts is an impediment to the production of accountability because it blocks access to crucial information for the accountability of results and, consequently, the effectiveness of a transparency that is based on events (inputs, outputs) and not on processes.
Transparency is crucial in the axis "Selection process of the SO": these are acts embodied in publications of Diário Oficial that initiate and end the entire process of negotiation of the management contract. At first, the acts on this process enable the transparency required to allow competitiveness in the choosing of the managing entity thus ensuring that the process occurs democratically.
The Public Call shows the criteria adopted for the selection process of the partner Social Organization of Culture, and the references needed for the preparation of the technical and budgetary proposal. The absence (or untimely publication) of its publication directly impairs the isonomic choice for an organization.
Regarding the axis "Documents related to the content of the contract", the number of opinions with notes on the non-compliance with the Fiscal Responsibility Law (Lei Complementar n.101 -Lei de Responsabilidade Fiscal [LRF], 2000) and with the budget-financial proposal is noteworthy: we note here that that there was no planning in the decision to delegate a given service to a SO. In the case of the LRF (2000), due to the disregard for the budgetary-financial impact and even the lack of compatibility and adequacy of contracts in relation to the Lei Orçamentária Anual (LOA -Annual Budget Law) and the Plano Plurianual (Multiannual Plan). Regarding the budget proposal, due to the Continue lack of expenditure forecast for the entire execution of the contract, or even the absence of approval of the proposal. Regarding the outputs of the contract -embodied in the analysis of the "account-giving" theme -, although the opinions on accounting and financial procedures are predominant, it is noteworthy that TCE-SP stressed the inefficiency of internal control, especially of the Evaluation Committee, indicating an important gap in obtaining accountability: the lack of integration between the action and production of information from internal control to the external one. This aspect adds to the fact that 73% of the sample of opinions on account-giving presented problems in the concluding opinion of SEC -the main output of the contract, which describes the results achieved, the economy achieved, and primarily, it contains the confirmation that the expenses were made and regularly accounted for and verified by internal control.
In this regard, we must highlight that irregularities about the performance of internal control (composed by: SEC, Board of Directors and Evaluation and Contract Management Unit) are pointed out in two ways by TCE-SP: both in the direct production of the report (output), and in the inefficiency of monitoring the execution of the contract (internal proceedings), whose influence on the outcome of the opinion is more diffuse since, as observed in the literature, it does not constitute a "transparency event".
In addition to the recurrence of notes, we must direct the analytical observation to the status of the opinion: 82% of the sample of the opinions on management contracts is composed of opinions with "regular with reservations" results (case in which the irregularity is not a liability to the Treasury and TCE-SP makes recommendations to reorganize the irregularities). However, when analyzing opinions trialed as "irregular", we observe the overlapping of notes, and the following were unanimous irregularities: the problems related to the publication of the draft contract, the disclosure of the organizations that manifested interest in engaging in the contract, and issues with the budget proposal and LRF (2000). This shows that the absence of transparency impaired the regularity of the contract, leading to sanctions by the external control.
We must highlight that incorrigible irregularities -the basis for the disapproval of the MC in four opinions -are directly related to transparency in the selection of a SO since an incorrigible irregularity is characterized by the absence of the publication of the public call or by issues in the draft of the management contract.
These flaws in the transparency of MC inputs represent difficulties for the future accountability of both the government and the SO for results -if not the impossibility of achieving accountability. The low transparency of inputs -as presented above -indicates that SEC has not been able to present nor ensure that the organizations will present all information on the regularity of qualification and selection of SO to the public.
Regarding opinions on the irregularity of account-giving, we verified that 90% of the sample is composed by the status "regular with reservations". In opinions considered "irregular", the reasoning of TCE-SP is based on irregularities in the internal activities of organizations -e.g., purchasing and hiring procedures -that were not corrected after justifications of those responsible. Such issues are not easily measurable and depend on the publication of manuals for purchases and contracting services, as well as the effective monitoring by internal control.
Such aspect increases the importance of integrating the whole information system -inputs and outputs -to achieve the accountability of results. Transparency in the qualification of SO, in the choice of the organization that will manage the contract, and in the results helps to identify the regularity in the conformity to the management contract. The consistency of internal control in monitoring and reporting, in turn, allows the identification of results, assisting external control. Without internal control information, external control needs to verify procedural issues and discover the origin of irregularities thus engaging in a kind of control more focused on procedures than on results.
We verified that, in the monitoring of MC and AG, the notes reveal that the absence of information has impacts on the accountability of results: regarding management contract, actions related to the selection of the organization and documents related to budget planning directly influenced the judgment of irregularity. On the other hand, in AC, the variety of notes indicates that documents responsible for demonstrating results do not fulfill this function -the reports and conclusive opinions of the control bodies do not ratify the results achieved or the regularity of the processes that underpinned them.
Finally, the lack of timely trial by the external control largely impairs the accountability for results: the average time for trial is longer than six years; that is, the trial is often conducted some years after the occurrence of the irregularity.

FINAL CONSIDERATIONS
The main conclusions of the analysis of the of the opinions of TCE-SP about the contractualization of results between the government and SO of culture of São Paulo are: 1. The most recurrent notes by TCE-SP are problems related to public convocation in the opinions of MC (which mean low transparency), failures in the conclusive opinion by the Executive branch (which mean low government capacity to hold the SO accountable), and failures in the outputs of account giving opinions (which mean low accountability of SO before the State, citizens and users of services); 2. There is a lack of transparency of the inputs of management contracts -especially in the choice of the organization and in the content of the contract to be signed; 3. There is very low accountability for the outputs because the judgment by TCE-SP occurs years after the execution of the services, and the reports do not indicate that the Executive branch takes actions to monitor, evaluate or charge for the results from the SO; 4. The accountability model is very institutionally fragmented and the internal and external controls are not integrated: the various instances of Executive control do not exercise effective control, given that several irregularities are identified in the judgment of TCE-SP, and such trial occurs only many years after the acts thus becoming ineffective to repair/correct and prevent irregularities; moreover, no integrated analysis is made between MC and AC by TCE-SP, generating a disconnection between transparency and accountability -the verification of lack of transparency in MG does not affect accountability for results); 5. The weakness of the internal control of the Executive branch and the untimeliness of external control make the accountability for results practically impossible, as well as the planning of the State about the future actions of SO.
The discussion conducted in this study emphasizes, as recommended by the literature, that the accountability for results presupposes that the information is produced to actually expose such results through the flow of inputs and outputs (Heald, 2006). For such, discussions on transparency must go beyond the provision of information to thus produce quality information about services and their results. The information must, at the very least, be timely, measurable, and capable of exposing those who are responsible so they can inform about the planning and formulation of policies by the government -which remains the responsible for the actions conducted by SO before society. Thus, this study points out the need to redefine the functions of control instances, to reduce the inefficiency of monitoring activities and, consequently, the of the verification of results.
The conclusions of this study also indicate that the flow of information required for accountability must be inserted in the corrective evaluation cycle, allowing the accountability and replanning of contracts and policies by the State. This issue has a deep relationship with accountability in the internal instances of the State and SO, which, if functioning properly, favor learning, the culture of transparency and, consequently, improve the planning of these organizations.
Finally, for future studies, we highlight the importance of analyzing the implementation of transparency from the internal point of view of organizations, focusing mainly on the structuring and improvement of the quality of information used in the monitoring and planning of services. Moreover, other arrangements beyond contractualization and that also involve the intersection between public and private should be studied, aiming at improving the services provided to society.