A group decision model for diagnosing barriers to achieving goals in a Brazilian Water Resources Program: an analysis of the state of Alagoas

Modelo de decisão em grupo para diagnóstico de barreiras ao alcance de metas no Programa Brasileiro de Recursos Hídricos: uma análise do estado de Alagoas

ABSTRACT

This aim of this article is to develop a conceptual group decision model capable of diagnosing existing barriers to achieving goals in results-based management in the Water Resource Management (WRM) system of the Brazilian state of Alagoas. The model was based on a Soft Operational Research method, incorporating systematic feedback on non-compliance targets and applied in the context of the transfer of funds from the federal government program to the WRM system in Brazilian states. This program involves the problem of multiple uses of water resources, and it is composed of variables in which actions are necessary to achieve the established goals. The model helped public water resource managers from Alagoas to understand problems related to non-compliance with the goals set by the program, directing management to the improvement of deficient processes. Consequently, we have better water management for its multiple uses, with society as the major beneficiary.

Keywords:
Water resource; Public policies; Results-based management; Problem structuring method; Soft Systems Methodology

RESUMO

Este artigo tem como objetivo desenvolver um modelo conceitual de decisão em grupo capaz de diagnosticar as barreiras existentes para o alcance de metas na gestão pública orientada por resultados no Sistema de Gestão de Recursos Hídricos (SGRH) do estado brasileiro de Alagoas. O modelo foi baseado em um método de pesquisa operacional soft, incorporando feedbacks sistemáticos sobre as metas de não conformidade, e aplicado ao contexto do repasse de recursos financeiros do programa do governo federal para o SGRH nos estados brasileiros. Este programa envolve o problema dos usos múltiplos dos recursos hídricos e é composto por variáveis nas quais são necessárias ações para atingir as metas estabelecidas. O modelo tem a capacidade de auxiliar os gestores públicos estaduais de recursos hídricos a compreenderem os problemas relacionados ao descumprimento das metas definidas pelo programa, direcionando a gestão para a melhoria dos processos deficientes. Consequentemente, melhorar a gestão da água em seus múltiplos usos, onde a sociedade é a maior beneficiária.

Palavras-chave:
Recursos hídricos; Políticas públicas; Gestão orientada por resultados; Método de estruturação de problemas; Soft Systems Methodology

INTRODUCTION

Water is an essential resource used as a raw material for many human activities, such as human consumption, irrigation, manufacturing, power generation, and transportation (Vera et al., 2017). Since the second half of the 20th century, urbanization processes, agricultural expansion, and industrialization have increased the demand for water resources (Silva et al., 2022). However, it is a scarce resource all over the world, especially in the regions most affected by intense and prolonged periods of drought. Even Brazil, considered a natural resource-rich country, has a poor geographical water distribution.

Public management of multiple water users is complex and demands numerous actions to mitigate conflicts that arise over water use and ensure a continuous supply to users. Furthermore, decision-making on this natural resource is highly site-specific, and is often intended to achieve political goals. Consequently, it is more likely to escalate into open conflict (Rus, 2014; Waters et al., 2021). Due to these facts, many problems related to water resources management can be attributed to a failure of governance rather than the availability of the resource itself (Bezerra et al., 2021). Therefore, in addition to the economic components, the performance measurement of public organizations must consider efficiency components (Gerasimova et al., 2019).

These growing pressures for reforms in public management practices make public organizations seek tools and strategies traditionally enshrined in the private sector (Aragão & Fontana, 2022). The same occurs in water resources management, where planners and managers have sought to adopt new management methods, such as results-based management (RBM) (Saghi-Jadid & Ketabchi, 2021). RBM is an appropriate model to increase the effectiveness of public management through the flexible management of processes and projects, aiming to eliminate the dysfunctions of the bureaucratic model (Ha & Hai, 2020). “RBM is understood as a management strategy meant to ensure that processes (inputs, activities) contribute to the achievement of desired results (outputs, outcomes, and impacts)” (Kirschke et al., 2022, p. 3). In the public sector, the result is the maximization of value creation for the public (Kanufre & Rezende, 2012).

The literature review demonstrates that the use of RBM is widespread in different contexts of public and not-for-profit organizations, such as in education performance management (high school or university institutions) (Borodiyenko et al., 2020; Bouchamma & April, 2020; Kamanzi et al., 2019; Kure et al., 2021; Segatto & Abrucio, 2017); health systems and/or humanitarian programs (Cordova-Pozo et al., 2018; Khan et al., 2020; Lainjo, 2019; Paulo, 2016; Witter et al., 2019), and national/state/city governance or government agencies (Gonzaga et al., 2017; Gwata, 2019; Kanufre & Rezende, 2012; Kwon et al., 2021; Peci et al., 2008; Porpino & Stefani, 2014).

With specific regard to Water Resource Management (WRM), Jumpa et al. (2013) studied the RBM of water resources to explain rural development with the Bhumibol dam and the Sirikit dam. Bongei & Kaburu (2021) assessed the effect of RBM on the sustainability of community water projects in Bomet County, Kenya. Saghi-Jadid & Ketabchi (2021) analyzed the indicators proposed by RBM through an optimization model based on MODFLOW and an evolutionary algorithm in the case of the Nandan (Iran) aquifer. Makanda et al. (2022) proposed a conceptual results-oriented policy monitoring framework to improve sustainable water resource utilization.

In Brazil, the Consolidation Programme for the National Pact of Water Management (PROGESTÃO), regulated by the National Water and Sanitation Agency (ANA), Resolution No. 379/2013 (Almeida et al., 2020) is based on the principle of financial reward through the achievement of targets. In other words, this program uses an RBM view in the allocation of federal public resources for water resources management in the states. Nevertheless, the state of Alagoas, for example, did not achieve the minimum level in the program variables, remaining below the pre-established goal. Understanding the barriers that prevented the targets from being reached is essential in strategic planning reorganization.

Therefore, this work aimed to develop a conceptual group decision-making model for diagnosing the barriers to achieving targets in a real-life context of result-based public management to improve the WRM system in the Brazilian state of Alagoas. For this purpose, public managers at the State Department for the Environment and Water Resources (SEMARH), directly responsible for managing PROGESTÃO in Alagoas, were interviewed.

The model uses a Problem Structuring Method (PSM), Soft Systems Methodology (SSM), considering a group of decision-makers and systematic feedback on non-compliance targets. PSMs are Soft Operational Research (SOR) tools used to understand complex decision environments, such as those that deal with several perspectives and/or people simultaneously, with conflicting objectives and situations of uncertainty (Feitosa & Carpinetti, 2022; Gomes Junior & Schramm, 2021; Smith & Shaw, 2019). The SSM, through an “open world assumption” view, does not accept the problem situation as presented. The SSM seeks to express the complex problem in a structured way, using for this a rich picture, root definitions, and conceptual models (Wang et al., 2015). Several authors have applied the SSM to different types of problems. In the water resources management context, we can highlight the works of Acero López et al. (2019), Bunch (2003), Gilbert & Pratt-Adams (2022), Gomes et al. (2015), Kayaga (2008), Souza Junior et al. (2019) and Suriya & Mudgal (2013). These studies have some shared characteristics, such as dealing with ill-defined or messy situations, a search for new solutions to problems involving multiple actors, and involving non-converging opinions or criteria.

However, the use of the SSM in result-based management situations is still a recent phenomenon. Thus, the conceptual model developed herein can help actors to understand the problem in an interactive and learning-oriented way. De Carvalho (2021) considered the absence of RBM one of the main barriers to innovation in public service in Brazil. Our model helps to gain a better understanding of the main challenges involved in achieving the program's targets and, as a result, provides important insights for managers to propose strategies and remedial measures. Furthermore, one of the contributions of this work is to meet Sustainable Development Goal (SDG) 6: Clean water and sanitation.

MATERIAL AND METHODS

This research can be classified as exploratory, descriptive, and qualitative. It is exploratory because it makes the problem more explicit, descriptive because the characteristics of the phenomenon are described, and qualitative because a qualitative approach is used to analyze the problem and present the results.

Research locus

The PROGESTÃO is a financial incentive program for state systems to be exclusively applied in actions of institutional strengthening and water resources management, through the achievement of goals that are defined based on the management complexity chosen by the federative unit (Agência Nacional de Águas e Saneamento Básico, 2021). Membership is voluntary and open to all states interested in collaborating to fulfil the goals of the National Pact for Water Management.

The program seeks to promote effective articulation between WRM and the regulation of water use. It is carried out at the national and state levels and aims to strengthen the Brazilian model of integrated, decentralized, and participatory water governance. The PROGESTÃO foresees four typologies (Table 1) to be chosen by the federation units (states) according to their current management complexity based on legal regulations.

Table 1
Typology according to the complexity of management in the PROGESTÃO.

Typology	A	B	C	D
Complexity	Low	Regular	High	Very High
Types of conflicts	No conflict. Watershed has punctual and dispersed uses.	There are conflicts over the use of water in critical sub-watersheds.	Greater coverage and intensity of conflicts over the use of water in the watershed.	Conflict over the use of water is highly complex and widespread throughout the watershed.
Institutional structure	- Basic monitoring;	- Improvement of monitoring in sub-watersheds or critical sections;	- Specific monitoring to follow up on management and framing goals;	- Water and usage billing agency.
	- Grants for selected significant uses;	- Grants for critical sub-watersheds with conflicts;	- Grants for the entire watershed;
	- Macro water balance and strategic planning studies;	- Water resources plans and framework for critical sub-watersheds;	- Water resources plan and framework for watersheds;
	- Instance of articulation between the states and the union.	- Sub-watershed committees where necessary.	-Watershed committees.

• Source: adapted from the Agência Nacional de Águas e Saneamento Básico (2021).

The evaluation performance of each Brazilian State within the chosen typology occurs through pre-established goals for the 31 variables of the program, organized into four groups (Table 2).

Table 2
State Management Variables in the program.

	Group		Variable
1	Institutional Organization	1	Institutional Organization of the Management Model
		2	Processes management
		3	Legal Framework
		4	State Water Resources Council
		5	Watershed Committees and Other Collegiate Bodies
		6	Water, Watershed or Similar Agencies
		7	Social Communication and Information Dissemination
		8	Training
		9	Articulation with User and Transversal Sectors
2	Planning Variables	1	Water Balance
		2	Hydrographic Division
		3	Strategic planning
		4	State Water Resources Plan
		5	Watershed Plans
		6	Watershed classification
		7	Special Management Studies
3	Information and Support Variables	1	Cartographic Base
		2	Registration of Users, Uses and Interferences
		3	Hydro-meteorological monitoring
		4	Water Quality Monitoring
		5	Information System
		6	Research, Development and Innovation
		7	Decision Support Models and Systems
		8	Critical Event Management
4	Operational Variables	1	Grant of Right of Use
		2	Inspection
		3	Billing
		4	Financial Sustainability of the Management System
		5	Water Infrastructure
		6	State Water Resources Fund
		7	Inductive Programs and Projects

• Source: adapted from the Agência Nacional de Águas e Saneamento Básico (2021).

The evaluation process analyzes the targets of each variable on a 5-point Likert scale (certification process). The number of financial resources allocated to each state participating in the program is the same, regardless of the typology chosen. However, the amount of implemented financial resources depends on the achievement of pre-established targets. Thus, variables with an evaluation below the target are considered a challenge, that is, they are critical variables and deserve greater attention from those involved. Therefore, the PROGESTÃO is classified as result-based public management because the annual amounts transferred to the states are proportional to the score obtained in the certification process. On the other hand, depending on the adopted typology, some goals can be mandatory or optional - in the latter case, unfulfilled goals do not affect the final certification grades, nor do they reduce the amount paid for achieving goals (Agência Nacional de Águas e Saneamento Básico, 2021).

In the state of Alagoas, Brazil, the management and execution of the PROGESTÃO is the responsibility of the State Department for the Environment and Water Resources (SEMARH). In the first cycle of the program (2013-2017), the framed typology by Alagoas was B. According to a report from the Institute for Applied Economic Research (Instituto de Pesquisa Econômica Aplicada, 2017), some goals established in this cycle were not achieved and others were underestimated. Thus, it was deemed that the classification was not the most appropriate. Thus, in the second cycle (2018-2021), Alagoas state was included in Typology C. Typology C involves a greater degree of complexity and challenges, as described in Table 1. However, it is up to the state to be more committed to its resources management to achieve the goals. Although this changing typology can attract more financial resources from the ANA, its complexity implies numerous variables that remain unfulfilled. This is because there are many barriers to managing the state's water resources, which justifies studies like this one.

Proposed conceptual model

The group decision model proposed here follows the Soft Systems Methodology (SSM) steps, developed by Checkland in 1972, considering the works of Abuabara et al. (2018), Silva & Fontana (2021) and Wang et al. (2015). Our model was divided into seven stages, numbered in Figure 1, adapting the SSM steps to a result-based public management situation by incorporating systematic feedback on non-compliance targets. This assists groups of managers in systemic learning and better planning for the allocation of public financial resources.

Figure 1
Group decision model based on the SSM for result-based public management.

The first stage (Complex Situation Inspection) consists of collecting information, where all the structures that make up the decision-making process in question are observed. In this stage, characteristics relevant to the problem are captured. In general, these involve elements of the structure (physical issues or limitations), elements of the process, norms, values, and organizational culture.

In the second stage (Expressing a Messy System), the intention is to investigate the main barriers to achieving goals in result-based public management. Here, a graphic representation, also known as a “rich picture”, is constructed, which allows an easier visualization of all the aspects of the messy situation. The rich picture is generated through workshops with a group of decision makers, conducted by a facilitator.

The third stage consists of formulating the root definitions, seeking to compile the human activities identified in six components, called CATWOE:

• C (customer) - users, victims and/or beneficiaries of the system’s output;

• A (actor) - the people who perform the activities of the system;

• T (transformation) - the change that the system brings about (inputs into outputs);

• W (Weltanschauung) - a worldview - the viewpoint that defines the activity of the system;

• (owner) - who can create, change or destroy the system;

• E (environment) - system restrictions.

The fourth stage (Formulating Conceptual Models) consists of defining a viable system for carrying out the activity, i.e., the “monitoring and control” components are built into the conceptual models. The SSM identifies three elements (3Es) to measure system performance:

• Efficacy (E1) -does the system produce the output it is supposed to? (Related to the “what” of the root definition);

• Efficiency (E2) - does the system produce the output it is supposed to? (Related to the “how” of the root definition);

• Effectiveness (E3) - does the system meet the goals and aspirations of the owner? (Related to the “why” of the root definition).

At this stage, it is important to check whether the decision makers are satisfied with the conceptual model. If they are not satisfied, the recommendation is to return to Stage 3.

The fifth step deals with comparing the real world (model expressed by the “rich picture”) with the conceptual model (Stage 4). The analysis of these differences is what generates discussions to propose solutions, changes, and actions. If there are viable and desirable changes in the model, the decision makers should be taken to Stage 6, where they will verify these viable and feasible changes, observing the economic-financial and organizational structure issues and those of the system as a whole.

Again, it is important to check whether the group of decision makers is satisfied with the changes made. If they are not satisfied, it is important to restart the debate (Stage 6). If they are satisfied, they should proceed to Stage 7, which deals with the implementation of these actions. In addition to the SSM stages, this conceptual model proposes a stage of monitoring the implementation of actions and gauging the achievement of goals. For as long as the goals are not met, the actions must be re-evaluated by the system. In this way, the management team will focus the improvement process on goals not yet achieved, optimizing the applied resources.

The SSM is suitable for the problem described in this article, as it is the effort to structure a problem, when we assume that the decision makers know the problem situation. On the other hand, there is a considerable degree of complexity. The SSM prompts the organization of information through representation by figures (rich picture), identification of the main elements of the situation from the key concepts (CATWOE) and using some driving questions such as “what are the difficulties?” or “what are the barriers?” to achieve a certain objective (generation of a conceptual map). Thus, it is also concluded that the results of the SSM are not a prescription, but a conclusion based on a more systemic way of thinking.

Data collection

The data were collected through face-to-face interviews with public managers at the SEMARH of Alagoas, Brazil who are directly responsible for managing the PROGESTÃO program in the state. Although the proposed methodology can include more decision makers, in the case under study, there are only three managers. They are operationally responsible for monitoring the targets and have wide experience in the field of water resources (proven experience of more than five years).

The questions focused on attempting to gauge the perspective of these managers regarding the non-compliance variables of the program and served as elements for further application of the SSM method. Analyzing every program variable is very complex for decision makers. Therefore, we suggested analyzing a group of variables (Table 2). We asked the decision makers for information on which group of variables should be analyzed first, and Group 2 (“Planning Variables”) was given priority. The entire process was conducted by the authors of this work, acting as analysts of the problem situation.

RESULTS

In the first stage, the managers were brought together to understand the problem. They reported that it was difficult to identify the barriers to achieving the program's goals because of a lack of understanding of the problems as a whole. Furthermore, they considered all the situations reported to be of extreme urgency or essential for efficient water resources management.

We then asked the following question: In your opinion, what are the main barriers to achieving the program's goals on Planning Variables? From these reports, a rich picture of the problem situation was built, represented in Figure 2 (Stage 2). This picture provides an overview of the relationships between the actors who work within this system and also helps to understand the roles of the department within this environment. Within the rich picture, one of the relationships between the ANA and SEMARH stands out, which is the management tool itself, as well as the relationships between the department and consumers, committees, and Alagoas state's own watersheds, highlighting the greatest difficulties currently encountered by the department.

Figure 2
Rich picture.

The rich picture is a reflection of how the current situation works. After its construction, the identification of key definitions of human activity within this context was facilitated, as summarized in CATWOE (Stage 3):

• C: Small and large watershed users and civil society for environmental protection;

• A: State Department for the Environment and Water Resources (SEMARH) and watershed committees;

• T: Several processes: In relation to the watersheds, carrying out water balance studies and defining watershed plans; In relation to customers, the analysis of requests for the granting of use;

• W: Variables and consequent typology (see Table 1);

• O: National Water and Sanitation Agency (ANA) of the federal government, and State Department for the Environment and Water Resources (SEMARH) of the state government;

• E: human, organizational and budgetary resources.

The inputs of the formulating conceptual model are the fundamental definitions of the problem (Stage 4). The concept map was built considering the activities necessary to meet the goals of the planning variables, which are highlighted in red in Figure 3.

Figure 3
Conceptual map.

The world view (W in CATWOE) is given by the classification of the state of Alagoas as Typology C. For this purpose, among the variables for which the goal was not achieved, it is necessary that they have the following status:

• 2.1. Water balance: Possesses adequate knowledge of the relationship between water supply and water demand under the state’s domain (surface and underground water), through specific studies or water resources plans;

• 2.5. Watershed plans: Plans in force in 50% to 75% of hydrographic management units;

• 2.6. Watershed classification: There are surface or underground water reservoirs framed in CONAMA resolutions 357/2005 and 396/2008, respectively (Conselho Nacional do Meio Ambiente, 2005, 2008);

• 2.7. Special management studies: Special and updated studies on some topics of interest to management in certain regions or watersheds are sufficient to guide management actions in the aspects covered by them.

The conceptual map (Figure 3) allows managers to view how the variables are related and the strategic elements for achieving the goals. Thus, the model also allowed the identification of other elements essential for achieving the goals that had not been achieved in the program. The measurement elements of the SSM model are already meant to evaluate and monitor the variables in the program. However, based on the analysis, it is necessary to prioritize the four elements highlighted in the conceptual model (Figure 3) in the drive to improve efficacy, efficiency, and effectiveness.

Following on, desirable and feasible changes must be evaluated (Stage 5) since, within the framework of goals established by the program and chosen typology, not all variables are established as priorities, despite their relevance in operational terms. In this case, the variables 2.2 Hydrographic Division, 2.3 Strategic planning, and 2.4 State Water Resources Plan were not mentioned because the variables had already been achieved in the current typology. Hence, the managers were satisfied with the conceptual model that was created.

DISCUSSION

For RBM efficiency, managers need to focus their attention on the regular measurement of variables, making adjustments to improve the efficiency and effectiveness of their programs to achieve objectives (Ba, 2021). The implementation and monitoring stage could not be followed by the authors. However, the conceptual map improved the managers' understanding of the problem and its variables, helping them to prioritize actions and related components. The identification of thematic axes obtained by our conceptual model can help conduct the program more effectively since the resources in the department are too scarce to achieve all the goals in the same period.

Although the variables are individually assessed by PROGESTÃO, the construction of the conceptual map shows a dependency relationship between them (Figure 3). For example, it is not possible to define the watershed classification without having already carried out the water balance, which, in turn, depends on water supply studies and special management studies that support water resource management.

On the other hand, socio-economic studies, water quality monitoring, and the establishment of watershed committees can be carried out in parallel. This characterizes another product of this study, which is the prioritization of the thematic axes, serving as a guide for the department's managers. Pubic management by RBM is challenging, as it requires greater responsibility on the part of organizations for definitive results, as well as demanding difficult decisions to guarantee the expected results (Bhattarai, 2020). The definition of the thematic axes/elements makes it possible to conduct the program more effectively.

In addition, it also allows us to identify that in the absence of the studies necessary for the planning goals, it is also not possible to establish plans for charging for water use, the development of projects for the conservation of natural resources, and a guarantee of their rational use or financial sustainability. These items are variables of the operational goals also established by the PROGESTÃO (Group 4). Therefore, poor performance with regard to the planning variables has a direct consequence on the performance assessed by the operational variables. Thus, the department is penalized twice in this assessment of the PROGESTÃO.

Failure to meet the established targets will then lead to financial penalties for the department, which already has scarce financial resources and labor, and does not receive resources from the ANA. Achieving the goals would be an alternative to guaranteeing resources to carry out operational activities and make management more efficient.

Rus (2014) affirmed that there is substantial case-study evidence of natural resource conflicts triggered by insensitivity on the part of local government. A ruler may choose to invest in either military repression or productive public goods (physical and social infrastructure) to solve the conflict over natural resources (Sarr & Wick, 2010; Waters et al., 2021). In other words, public policy choices may shape the development outcomes in natural resource-rich countries. Thus, we reinforce here that the decision-making in the public sector, especially on the distribution and use of water resources, must be scientifically based so that society trusts the government and demystifies the idea of manipulation in the decision-making process.

CONCLUSIONS

Results-based management is an important element for conducting WRM. In this respect, the PROGESTÃO is based on the principle of financial reward through the achievement of targets. These financial resources are crucial for WRM in Brazilian states.

Despite the program's weaknesses, the implementation of the PROGESTÃO in Alagoas stimulated the planning, monitoring, and control of water management in the state. However, failure to achieve the desired goals of the program implies lower funding from the ANA, which affects the daily activities of the department and the water management system itself.

The proposed conceptual model successfully structured the problem and established thematic axes to guide managers in understanding the barriers to achieving targets, supporting decision-making on prioritizing resource allocation, and defining new goals and objectives.

The improvement of the targets in these priority axes contributes to the full transfer of funds from the ANA to be used in the improvement of the state management system and encourages improvements for the users of the system, resulting in a social contribution.

The main contributions of this work are associated with the proposition of a group decision model to support public financial resource allocation in results-based public management situations. Our proposal, through the stages of the SSM and incorporated systematic feedback, is a clearer, more transparent, and grounded scientific process to understand problems related to non-compliance with the set targets, define actions to solve these problems and monitor the targets that have not yet been achieved. As a consequence of these actions, society benefits from tactical and operational public management focused on improving unachieved targets, optimizing the financial resources used, and improving the public services offered.

Although the proposed model can incorporate different perspectives of the problem, a limitation of the real problem that was presented was that it did not consider the opinions of users, civil society, and local academic bodies.

ACKNOWLEDGEMENTS

The authors would like to thank the Coordination for the Improvement of Higher Education Personnel (CAPES) - Finance Code 001, the Universidade Federal de Pernambuco (UFPE) - Edital pesquisador PROPG nº 06/2022 (Marcele Elisa Fontana), and the Universidade Federal de Alagoas (UFAL).

REFERENCES

• Abuabara, L., Paucar-Caceres, A., Neyra Belderrain, M. C., & Burrowes-Cromwell, T. (2018). A systemic framework based on Soft OR approaches to support teamwork strategy: an aviation manufacturer Brazilian company case. The Journal of the Operational Research Society, 69(2), 220-234.
• Acero López, A. E., Ramirez Cajiao, M. C., Peralta Mejia, M., Duran, L. F. P., & Diaz, E. E. E. (2019). Participatory design and technologies for sustainable development: an approach from action research. Systemic Practice and Action Research, 32(2), 167-191. http://dx.doi.org/10.1007/s11213-018-9459-6
  » http://dx.doi.org/10.1007/s11213-018-9459-6
• Agência Nacional de Águas e Saneamento Básico - ANA. (2021). PROGESTÃO - Programa de Consolidação do Pacto Nacional pela Gestão das Águas Retrieved in 2021, October 27, from https://progestao.ana.gov.br/
  » https://progestao.ana.gov.br/
• Almeida, B. M. J., Fontana, M. E., & Levino, N. A. (2020, December 02-04). Programa de Consolidação do Pacto Nacional pela Gestão das Águas (PROGESTÃO): uma avaliação qualitativa das experiências administrativas estaduais no Nordeste. In X Congresso Brasileiro de Engenharia de Produção (CONBREPRO), Online.
• Aragão, J. P. S., & Fontana, M. E. (2022). Guidelines for public sector managers on assessing the impact of outsourcing on business continuity strategies: a Brazilian case. Journal of Global Operations and Strategic Sourcing Ahead-of-print.
• Ba, A. (2021). How to measure monitoring and evaluation system effectiveness? African Evaluation Journal, 9(1), a553.
• Bezerra, A. P., Vieira, Z. M. D. C. L., & Ribeiro, M. M. R. (2021). Water governance assessment at different scales: a reservoir case study in the Brazilian semiarid region. Revista Brasileira de Recursos Hídricos, 26, e14.
• Bhattarai, R. N. (2020). Basic concepts and approaches of results based management. Journal of Population and Development, 1(1), 156-171.
• Bongei, M., & Kaburu, K. (2021). Result oriented approach and sustainability of community water projects in Bomet County, Kenya. International Journal of Innovative Research and Advanced Studies, 8(6), 40-47.
• Borodiyenko, O., Malykhina, Y., Kalenskyi, A., & Ishchenko, T. (2020). Economic, psychological and pedagogical preconditions of implementation of result-based management. Financial and Credit Activity: Problems of Theory and Practice, 2(33), 535-546.
• Bouchamma, Y., & April, D. (2020). The professional learning community to implement the Results-Based Management approach (RBM) in Québec. Canadian Journal of Educational Administration and Policy, 192, 77-85.
• Bunch, M. J. (2003). Soft systems methodology and the ecosystem approach: a system study of the cooum river and environs in Chennai, India. Environmental Management, 31(2), 182-197.
• Carvalho, A. B. R. B. (2021). Desafios da inovação na gestão pública brasileira. Revista Eletrônica Ciência & Tecnologia Futura, 1(2), 98-105.
• Conselho Nacional do Meio Ambiente - CONAMA. (2005, March 17). Resolução CONAMA nº 357, de 17 de março de 2005. Diário Oficial [da] República Federativa do Brasil, Brasília. Retrieved in 2023, January 28, from https://www.icmbio.gov.br/cepsul/images/stories/legislacao/Resolucao/2005/res_conama_357_2005_classificacao_corpos_agua_rtfcda_altrd_res_393_2007_397_2008_410_2009_430_2011.pdf
  » https://www.icmbio.gov.br/cepsul/images/stories/legislacao/Resolucao/2005/res_conama_357_2005_classificacao_corpos_agua_rtfcda_altrd_res_393_2007_397_2008_410_2009_430_2011.pdf
• Conselho Nacional do Meio Ambiente - CONAMA. (2008, April 03). Resolução CONAMA nº 396, de 3 de abril de 2008. Diário Oficial [da] República Federativa do Brasil, Brasília. Retrieved in 2023, January 28, from http://portalpnqa.ana.gov.br/Publicacao/RESOLU%C3%87%C3%83O%20CONAMA%20n%C2%BA%20396.pdf
  » http://portalpnqa.ana.gov.br/Publicacao/RESOLU%C3%87%C3%83O%20CONAMA%20n%C2%BA%20396.pdf
• Cordova-Pozo, K., Hoopes, A. J., Cordova, F., Vega, B., Segura, Z., & Hagens, A. (2018). Applying the results based management framework to the CERCA multi-component project in adolescent sexual and reproductive health: a retrospective analysis. Reproductive Health, 15(1), 1-13.
• Feitosa, I. S. C. S., & Carpinetti, L. C. R. (2022). Problem structuring combined with sentiment analysis to product-service system performance management. In Science and Information Conference (pp. 322-339). Cham: Springer.
• Gerasimova, L. N., Mezentseva, T. M., Parasotskaya, N. N., & Dvoretskaya, V. V. (2019). Features of environmental cost management. Amazonia Investiga, 8(20), 609-615.
• Gilbert, J., & Pratt-Adams, S. (2022). Systems thinking and soft systems methodology. In J. Gilbert & S. Pratt-Adams (Eds.), Soft systems methodology in education (pp. 3-34). Cham: Springer.
• Gomes Junior, A. D. A., & Schramm, V. B. (2021). Problem structuring methods: a review of advances over the last decade. Systemic Practice and Action Research, 35(1), 55-88.
• Gomes, S. F., Andrade, A. L., & Morais, D. C. (2015). Using soft systems methodology on the problem of water scarcity. In IEEE International Conference on Systems, Man, and Cybernetics (pp. 273-278). New Jersey: IEEE.
• Gonzaga, R. P., Frezatti, F., Ckagnazaroff, I. B., & Suzart, J. A. D. S. (2017). Avaliação de desempenho no governo mineiro: alterações dos indicadores e metas. Revista de Administração Contemporânea, 21, 1-21.
• Gwata, R. G. (2019). Original paper Results Based Management (RBM) as a tool for good governance. WORLD (Oakland, Calif.), 6(1), 1-8.
• Ha, P. N., & Hai, N. T. H. (2020). Facilitating results-based planning in developing countries: the case of vietnam’s ministry of agriculture and rural development. Public Administration, 5, 59-80.
• Instituto de Pesquisa Econômica Aplicada - IPEA. (2017). Programa de consolidação do pacto nacional pela gestão das águas: referencial básico de avaliação Rio de Janeiro: IPEA. Retrieved in 2021, October 27, from https://repositorio.ipea.gov.br/handle/11058/8150
  » https://repositorio.ipea.gov.br/handle/11058/8150
• Jumpa, S., Sakkatat, P., Prokati, W., Punyanuwat, A., & Pothisuwan, R. (2013). Results based water resources management for rural development of Bhumibol and Sirikit Dams in Thailand. Warasan Wichai Lae Songsoem Wichakan Kaset, 30(1), 59-68.
• Kamanzi, P. C., Lapointe, P., & Dembélé, M. (2019). The implementation of results-based management in quebec: between school principals’ optimism and teachers’ skepticism. Swiss Journal of Sociology, 45(3), 427-445.
• Kanufre, R. A. M., & Rezende, D. A. (2012). Princípios da gestão orientada para resultados na esfera municipal: o caso da prefeitura de Curitiba. Revista ADM, 47(4), 638-652.
• Kayaga, S. (2008). Soft systems methodology for performance measurement in the Uganda water sector. Water Policy, 10(3), 273-284.
• Khan, K. J., Begum, N., & Razak, A. (2020). Practicing results based management for enhancement of quality and performance of social development and humanitarian programmes. Universal Journal of Management, 8(5), 231-253.
• Kirschke, S., Avellán, T., Benavides, L., Caucci, S., Hahn, A., Müller, A., & Rubio Giraldo, C. B. (2022). Results‐based management of wicked problems? Indicators and comparative evidence from Latin America. Environmental Policy and Governance In press.
• Kure, N., Nørreklit, H., & Røge, K. M. (2021). Objective and results‐based management of universities: constructing reality or illusions? Financial Accountability & Management, 37(2), 204-230.
• Kwon, Y., Lee, J., & Park, C. (2021). Korea’s aid agencies and result-based management system. KIEP Research Paper, World Economy Brief, 11(4), 21-44.
• Lainjo, B. (2019). Results Based Management (RBM): an antidote to program management. JABS: Journal of Applied Biological Sciences, 5(1), 47-64.
• Makanda, K., Nzama, S., & Kanyerere, T. (2022). Assessing the role of water resources protection practice for sustainable water resources management: a review. Water (Basel), 14(19), 3153.
• Paulo, L. F. A. (2016). Planejamento estratégico e gestão por resultados: o caso do Ministério da Saúde. Physis (Rio de Janeiro, Brazil), 26, 981-1007.
• Peci, A., Figale, J., Oliveira, F. D., Barragat, A., & Souza, C. (2008). Oscips e termos de parceria com a sociedade civil: um olhar sobre o modelo de gestão por resultados do governo de Minas Gerais. Revista de Administração Pública, 42, 1137-1162.
• Porpino, G., & Stefani, E. (2014). O caso do PAC Embrapa: requisitos de uma gestão orientada para resultados no setor público. Revista de Administração Pública, 48, 343-366.
• Rus, H. A. (2014). Corruption, conflict and the management of natural resources. Economics of Governance, 15(4), 355-386.
• Saghi-Jadid, M., & Ketabchi, H. (2021). Result-based management approach for aquifer restoration problems using a combined numerical simulation-parallel evolutionary optimization model. Journal of Hydrology (Amsterdam), 594, 125709.
• Sarr, M., & Wick, K. (2010). Resources, conflict and development choices: public good provision in resource rich economies. Economics of Governance, 11(2), 183-205.
• Segatto, C., & Abrucio, F. (2017). A gestão por resultados na educação em quatro estados brasileiros. Revista do Serviço Público, 68(1), 85-106.
• Silva, A. C. D., Ferreira, E. D. C., Cabral, J. J. D. S. P., & Azevedo, J. R. G. D. (2022). Determination of hydrological stress in a river basin in northeastern Brazil. Revista Brasileira de Recursos Hídricos, 27, e2.
• Silva, W. D. O., & Fontana, M. E. (2021). Integrative multi-attribute negotiation model to define stakeholders’ responsibilities in the reverse flow channel. Journal of Cleaner Production, 279, 123752.
• Smith, C. M., & Shaw, D. (2019). The characteristics of problem structuring methods: a literature review. European Journal of Operational Research, 274(2), 403-416.
• Souza Junior, C. B., Koch, H., Siegmund-Schultze, M., & Köppel, J. (2019). An exploratory scenario analysis of strategic pathways towards a sustainable electricity system of the drought-stricken São Francisco River Basin. Energy Systems, 12, 563-602.
• Suriya, S., & Mudgal, B. V. (2013). Soft systems methodology and integrated flood management: a study of the A dayar watershed, C hennai, India. Water and Environment Journal: the Journal / the Chartered Institution of Water and Environmental Management, 27(4), 462-473.
• Vera, L. H. A., Montenegro, S. M. G. D. L., & Silva, S. R. D. (2017). Performance of water usage charge in the Nation’s domain as a water resource management tool in the São Francisco River basin. Revista Brasileira de Recursos Hídricos, 22, e7.
• Wang, W., Liu, W., & Mingers, J. (2015). A systemic method for organisational stakeholder identification and analysis using Soft Systems Methodology (SSM). European Journal of Operational Research, 246(2), 562-574.
• Waters, S., El Harrad, A., Bell, S., & Setchell, J. M. (2021). Decolonizing primate conservation practice: a case study from North Morocco. International Journal of Primatology, 43, 1046-1066.
• Witter, S., Chirwa, Y., Chandiwana, P., Munyati, S., Pepukai, M., & Bertone, M. P. (2019). The political economy of results-based financing: the experience of the health system in Zimbabwe. Global Health Research and Policy, 4(1), 1-17.

Edited by

Editor-in-Chief: Adilson Pinheiro

Associated Editor: Fernando Mainardi Fan

Publication Dates

• Publication in this collection
  13 Feb 2023
• Date of issue
  2023

History

• Received
  01 Aug 2022
• Reviewed
  16 Dec 2022
• Accepted
  14 Jan 2023