A general methodology for adaptative planning of urban water systems under deep uncertainty

Planejamento adaptativo de sistemas hídricos urbanos no contexto de incertezas profundas: proposta metodológica

ABSTRACT

Urban Water Systems (UWS) operate under a broad list of uncertainties. They usually depend on large amount of investments requiring long-term planning for a future full of changes with high degree of uncertainties (climate, social and economic). Many of these uncertainties do not have analytical representation neither they count on agreement among experts or decision makers concerning their influence in the performance of the UWS. The literature has named these changes as deep uncertainties (DU). This work presents a general approach to incorporate the influence of DU on planning and management processes of three types of UWS: 1) Water Supply Systems; 2) Drainage Systems and 3) Rainwater Harvesting Systems. The proposed framework defines steps toward the selection of the best policies and their evaluation in a broad set of scenarios. The particularities of each urban system led to adjustments in some steps of the general methodology. The approach proposed in this work was applied to a practical case, the Rainwater Harvesting Systems in the city of Ipameri, located in the State of Goiás, Brazil. The results highlight the impacts of DU factors on the system performance and reinforce this type of approach as a contribution towards adaptive planning for UWS.

Keywords:
Deep uncertainties; Adaptive planning; Urban water systems

RESUMO

A operação de Sistemas Hídricos Urbanos (SHU) é permeada por incertezas, e frequentemente depende de grandes montantes de investimento, além de exigir o planejamento a longo prazo em contextos futuros com mudanças constantes e alto grau de imprevisibilidade (mudanças climáticas, fatores sociais e econômicos). Vários desses fatores de incerteza carecem de representações analíticas, tampouco contam com amplo consenso entre especialistas acerca da influência que exercem na performance de SHU. A literatura internacional se refere a esse tipo de incerteza como incertezas profundas (DU). Este trabalho propõe uma abordagem generalista para incorporar a influência de incertezas profundas na gestão e planejamento de três categorias de SHU: 1) Sistemas de Abastecimento de Água; 2) Sistemas de Drenagem; 3) Sistemas de Aproveitamento de Água de Chuva. A abordagem proposta define etapas para seleção das políticas mais adequadas para cada sistema e para avaliação dessas estratégias em uma ampla gama de cenários. As particularidades de cada sistema levam a ajustes na estrutura geral da abordagem proposta. A abordagem foi aplicada para um Sistema de Aproveitamento de Água de Chuva na cidade de Ipameri no estado de Goiás, Brasil. Os resultados destacam a influência dos fatores de DU no desempenho dos sistemas e atestam a contribuição da metodologia para o planejamento adaptativo de SHU.

Palavras-chave:
Incertezas profundas; Planejamento adaptativo; Sistemas hídricos urbanos

INTRODUCTION

Urban areas in the 21st century are facing important challenges to maintain and expand infrastructure and sanitation services for their population. Urban water systems (UWS) face constant changing conditions in economic, environmental and social aspects. Variability in population growth, informal urban land occupation, water and soil pollution, climate changes, hydrologic extreme events and socioeconomic disparities are some of the factors that pressure urban water system operation and long-term planning (Zhang et al., 2019).

These issues are particularly acute for UWS in large cities of developing countries usually characterized by income disparities, rapid growth and widespread informality occupation, followed by weak management of urban services (Carrera et al., 2018). In this scenario, it has been difficult for many UWS to achieve their primary objectives, usually affecting the lives of the most vulnerable in society.

Many big cities in Brazil are examples of this historical process and face conditions that challenge the provision of acceptable sanitation and water services to all its inhabitants (Marengo, 2014). Some Brazilian metropolitan areas not rarely face extreme events such as severe floods, as in Petropolis in 2021 and Recife in 2022, but also droughts, as the 2016-2018 water crisis in the Federal District of Brazil, the 2014 water shortage in São Paulo and frequent droughts in large cities in the Northeast region (Fortaleza, Campina Grande and Recife).

Presuming that this is strictly the result of climate changes seem to be a fragile assumption, given these are water systems subject to different sources of uncertainties (climate and land use changes, increasing water demand, political and social choices). Therefore, management and planning strategies that do not consider these features tend to fail in dealing with unpredicted conditions and scenarios (Walker et al., 2013a; Marchau et al., 2019) resulting in lower performances and higher risks and costs to governments and communities.

Traditionally, decision support tools have focused on searching for the most likely future scenario to develop an optimal plan or management strategy that perform well under this most plausible scenario. Although this traditional approach has already provided many improvements in the water system analysis, it relies on the hypothesis that uncertainties result from a lack of information or random variation and can be addressed by constantly gathering information about the system (in the first case) or relying on statistical analysis or stochastic models (in the second case) (Walker et al., 2013a). However, recent understanding of these water systems recognizes them as characterized by conditions, parameters and/or system boundaries in which decision makers and planners do not know or cannot agree upon their probability distributions and other key aspects (Marchau et al., 2019). In these systems, traditional planning and management tools tend to be vulnerable to unlikely scenarios. These uncertainty factors or elements are defined in the literature as deep uncertainties (DU) given they present a high degree of uncertainty (Walker et al., 2003, 2013b; Lempert et al., 2006; Kwakkel et al., 2013; Marchau et al., 2019; Lempert, 2002). Usual examples of DU factors affecting urban water systems are related to economic and social aspects of future scenarios such as the increase in water demand, effectiveness of policies and regulation, water tariffs, time to obtain environmental licenses and water use permits, infrastructure investment rates and many other uncertain conditions (Walker et al., 2001; Zeff & Characklis, 2013; Trindade et al., 2017, 2020; Watson & Kasprzyk, 2017; Trindade, 2019; Giacomazzo, 2020).

In order to deal with these uncertainties, a set of methods has evolved as Decision Making Under Deep Uncertainty (DMDU) approaches, and they focus on reducing the vulnerability of policies and strategies to surprising developments or very uncertain futures. These methods change focus from determining the most likely scenario through the best predictive model and relying on it to plan the system operation, to exploring a multitude of future scenarios defined as a set of States of the World (SOW) and searching for policies that perform well, avoiding system failures in extreme conditions. The DMDU methods emerged from the Assumption Based Planning paradigm and differ from one another in terms of the way they deal with uncertainties and their influence in policies performance and in the decision making process. Examples of these methods are Exploratory Modeling (EM), Scenario Discovery (SD), Robust Decision Making (RDM), Multiobjective Robust Decision Making (MORDM), Dynamic Adaptive Planning (DAP), Dynamic Adaptive Policy Pathways (DAPP) (Marchau et al., 2019).

Despite the advantages of adaptive planning, DMDU approaches have yet to be largely explored in the planning and management of UWS in Brazil. Considering the recent changes imposed by the New Water and Sanitation Regulation and the usual uncertainties involved in UWS planning processes, one may expect that the definition of robust policies in the management of UWS could benefit from the support of DMDU approaches.

This work aimed to present recent evolution of DMDU principles and its emerging techniques in the literature and to propose an initial framework to build adaptive planning applicable in three types of UWS: Urban Water Supply Systems (UWSS), Urban Drainage Systems (UDS) and Urban Rainwater Harvesting Systems (URHS). The current framework is a result of a careful interpretation of the DMDU literature, and simultaneously aggregates the following aspects: 1) A change of focus from trying to predict the future, to exploring as many scenarios as possible and identify which conditions can lead to system failure; 2) The application of a new modelling paradigm based on preparing planning portfolios to adapt to any future, by monitoring how reality evolves and allowing adaptations to its outcomes (instead of planning one strategy to the most likely future); 3) The embedding of risk management based on risk of failure metrics, which are constantly monitored and whose thresholds, if crossed, trigger management actions; 4) The selection of management options based on their performance throughout multiobjective evaluation and optimization; 5) The incorporation of deep uncertainties (including economic and social uncertainties) into planning and management processes. Despite the fact that these features are already considered in modern international literature, the framework proposed in this work is innovative, since it is fully oriented to Brazilian water systems and is adjustable to systems of different natures (Urban Water Supply System - UWSS, Urban Drainage System - UDS and Urban Rainwater Harvesting System - URHS).

MATERIAL AND METHODS

In the last decade, modeling approaches have evolved to support adaptive planning that results in policies that perform well or at least acceptably in a changing world, represented as a large ensemble of scenarios (SOW). A common goal in these approaches is to find strategies that reduce the system vulnerability to uncertain and unpredictable future scenarios (Marchau et al., 2019). The idea is to change the problem question from “what will the future be like?” to “what actions or policies would be more appropriate to deal with a larger ensemble of plausible scenarios?”. Some studies have already applied these new methods in public transport planning (Agusdinata et al., 2006), water supply systems (Characklis et al., 2006; Kasprzyk & Reed, 2009), and flood control systems facing global changes (Groves et al., 2013; Bonzanigo et al., 2018; Casal-Campos et al., 2018; Marchau et al., 2019).

In the present work, we present a methodology to adapt a decision making process under deep uncertainty to urban water resources systems of three different natures. Initially we present an overview of the DMDU approaches using a bibliometric analysis to show how these approaches have evolved in recent years, summarizing their contribution to the decision making process. Then, we stablish general steps of DMDU modeling for UWS and finally we customize the general approach in order to support decision making for three categories of UWS. Figure 1 presents an overview of the methodology used in this study aiming to incorporate deep uncertainty influence in the planning and management process of urban water systems. In this Figure, steps II to VII represent the framework proposed in this study focused on applying DU analysis in Brazilian water systems.

Figure 1
Overview of the methodological scheme proposed in this study aiming to consider deep uncertainties in the planning and management of UWS. Steps II to VII represent the framework proposed to enable DU analysis into Brazilian water systems.

Overview of DMDU research evolution

A bibliometric analysis is a quantitative statistical technique used for literature mapping (Yu et al., 2022). Díez-Herrero and Garrote (2020) and Donthu et al. (2021) highlight the advantages of this technique over traditional reviews, based primarily on citations, although citations usually need time to accumulate (Zhang & Chen, 2020). Donthu et al. (2021) remind that this metric responds weakly to the newest research, but it is useful for new researchers to discover trends and to obtain the top classic articles. The overview of research literature in decision support under deep uncertainty is addressed in three steps: 1) data selection; 2) bibliometric data extraction and 3) mapping. In the first stage, the scope covers records in the last two decades from the Web of Science (WoS) database using the computational tool VOSviewer.

Based on an extensive review of DMDU literature and its interpretation, a framework was built to apply the main DMDU principles altogether in one single planning process layout specifically suited for Brazilian water system contexts. Based upon the fact that DMDU applications on Brazilian systems are still very limited, the proposition of a framework that encompass DU principles and the adaptation for Brazilian systems consists of an innovative work. The following sections describe the steps of the general framework and its application to each of the urban water systems according to their specific characteristics.

General definition of the System Simulation Settings

The first step in this framework is to define system settings, including all important design parameters, spatial network of infrastructure components and connections (reservoirs, treatment units, catchments, wastewater discharges), spatial distribution of water demands, environmental regulations and other operational aspects that are relevant to system management.

Objectives definition

After defining UWS settings, it is time to define the goals that society and water regulators expect to achieve with water system operation. These are the society’s expectation from the investment in the UWS and need to be maximized or minimized during the system life cycle or long-term modeling simulation. These objectives guide the UWS performance evaluation. This stage of the methodology can be enhanced through expert consultation and community involvement in the decision process and modeling exercises. The literature illustrates some usual objectives that are minimized in UWS modeling such as investment costs (Trindade et al., 2017; 2019; Herman et al., 2014), operational and maintenance costs (Zeff et al., 2014), risk of failures (Kwakkel et al., 2016), peak flow volume (Gold et al., 2019), consumption restriction frequency. Other objectives are maximized, such as average community income and reliability (Bartholomew & Kwakkel, 2020; Giacomazzo, 2020).

Identifying Deep Uncertainties (DU)

The next step of the framework consists in the identification of conditions that have the potential to be considered as deep uncertainties. These are conditions that decision makers know nothing or very little about, making it impossible to define probability distributions, boundaries or detailed behavior. Even knowing so little about these conditions, they are still important because they have the potential to influence the system and policy performance and their ability to reach the objectives. Usually, DU factors are data or information related to social and economic aspects of the UWS operation. The interaction between hydrologic and human systems usually involves uncertainties that are hard to represent in analytical terms. These uncertainties are clear candidates to be represented as DU factors. Their influence in the UWS operation may vary from one system to another. Some examples of DU factors in the literature are: tariff of the UWS service, effectiveness of management actions or regulations, increase or decrease in water demand rates, discount rate of infrastructure investments, time to get water use permits and environmental licenses (Kwakkel et al., 2014; Gold et al., 2019; Trindade, 2019; Trindade et al., 2019, 2020; Giacomazzo, 2020).

Building States of the World (SOW)

The SOW is the set of conditions that define the multiple contexts of the UWS long-term operation. It may combine climate, economic and social variables that can affect system performance and change their parameters from one possible future to the other. It is important to distinguish these data from the system simulation settings, which consists of permanent properties of the system that make it unique. Thus, if a certain technical feature presents multiple options of application that can change in different alternative futures, it can be incorporated to SOW definitions. Otherwise, if it is inherent to the system and can’t change with future possibilities, it is considered as a system feature and doesn’t embed SOW definitions.

Figure 2 illustrates an example of SOW definition in which each set of well-characterized uncertainties is paired to a set of DU factors. The SOW are usually comprised of well-characterized uncertainties (WCU), represented by the larger boxes on the left of Figure 2, and deeply uncertain conditions, represented by the smaller boxes on the middle of Figure 2. The well-characterized uncertainties represent data which probability distributions are known. Some studies consider climate data (precipitation, streamflow and evaporation) as well-characterized uncertainties (Trindade et al., 2017, 2019; Gold et al., 2019; Giacomazzo, 2020). On the other hand, the DU factors – defined in the previous section – are the second part of the SOW. The set of DU factors results from a sampling procedure that selects the values of the factors from a predefined range of possibilities. Figure 2, on the right side, also shows the objectives, which are calculated considering policy performance across all SOW. The objectives obtained are later used to define the robustness of each alternative under analysis (Trindade et al., 2017).

Figure 2
Description of States of the World (SOW) formulation and integration to the modeling exercise modified from Trindade et al. (2019).

An example of SOW construction is in the study of Trindade et al. (2017). The authors applied exploratory modelling to identify robust water portfolios to reduce drought vulnerabilities for a water system in North Carolina, United States. The study defined the SOW as composed of one WCU, which was a streamflow time series for each reservoir considered, as defined in previous studies, and one Latin Hypercube Sampling (LHS) sampled vector comprising 13 different DUs among the categories of demand, capacity, and costs.

Definition of the set of successful policies

A policy is a set of management actions or rules that are triggered at specific moments during the system life cycle, represented in the simulation. In some cases, a policy is comprised of short and long-term management actions or it may represent some specific setting of the system that changes the way it operates. This definition of policy may be adapted according to the UWS nature and particularities. But usually, the simulation analysis is interested in defining these policies and selecting the ones that generate better performances of the UWS under the context of an ensemble of SOW. The system performance is evaluated according to selected objectives of the system. Decision makers, managers and communities choose the required performance criteria (minimum or maximum values of the selected objectives). A policy is selected if it satisfies the predefined criteria when the system is simulated under the context of the ensemble of SOW. The policy will maximize or minimize predefined objectives, and its success or failure to do so will determine how preferable this policy is, compared to others.

Urban Water System analysis under DU context

A final step in the general framework presented in this work consists of defining an analysis of the system in the context of deeply uncertain conditions. This can include selecting policies, evaluating the tradeoffs among the system objectives, building infrastructure pathways and evaluating policy robustness (meaning an acceptable performance over a wider range of SOW). Another interesting exercise is to analyze the distinctions between performances of policies when incorporating DU into possible future scenarios, since it can express the potential of DMDU perspectives in supporting decision making and planning processes in complex water systems.

Another interesting alternative in this step is to apply multiobjective evolutionary algorithms in an exploratory search, instead of evaluating a pre-specified set of policies. In this case, multiple alternatives of short/ and long-term actions and risk tolerances are combined, building and exploring a multitude of policies in an optimization process. However, this application depends on the computational resources available for each case, which can be operationally detrimental to its execution.

RESULTS AND DISCUSSIONS

This section illustrates the adaptation of the DMDU framework built as an interpretation of DMDU principles and approaches proposed in Zeff et al. (2014), Gold et al. (2019) and Trindade et al. (2019) to urban water systems of three different configurations: 1) Urban Water Supply Systems (UWSS), 2) Urban Drainage Systems (UDS) and 3) Urban Rainwater Harvesting Systems (URHS). The adaptation considers specific details and requirements of each urban water system while selecting a set of robust decision in the context of deeply uncertain conditions.

DMDU research field overview

The overview of recent evolution of DMDU research field resulted from a bibliometric analysis conducted on October 19th, 2022, at 3:50 pm, employing the keywords "decision making", "deep uncertainty" and "water" to search the “title of a research article”, “abstract” or “keywords”. A filter applied to restrict the publication results from the years 2002 to 2022 resulted in a sample of 235 publications that met the selected criteria. The following analysis used the VOSviewer version 1.6.18.

Figure 3 shows the increase in the number of articles published over time illustrating significant upward trend beginning in 2010. The results show that 1,001 authors from 55 countries published 235 articles. The graph suggests that in the first decade of this century, more precisely from 2002 to 2010, research in this area was under development. In fact, 76.60% of the research is condensed over the last six years (2016-2022). This insight seems to suggest the recent and growing international research interest in the topic. However, the number of publications in 2021 and 2022 (under 30) suggests that there is a need to expand the research in this area to reach its potential to contribute to adaptive planning under the context of global changes expected in the near future.

Figure 3
DMDU publications per year (2002-2022).

The United States concentrate the publishing efforts, reaching almost 35% of the overall papers published from 2002 to 2022 in the WoS journals, followed by Australia (16%), China (15.75%), England (14%) and Netherlands (11%).

Figure 4 shows the co-occurrence map built with 72 keywords indicated by the authors, journals and publishers at least twice in the search, between years 2016 and 2020. The map also illustrates some keywords highlighted from 2016 to 2020. The set of keywords in the map provides an initial insight into the study fields and applications explored in most researches.

Figure 4
Co-occurrence map with 72 keywords and temporal analysis of publications (2016-2020).

From the bibliometric analysis, it was possible to highlight some approaches that have extensive applications for water resources systems, being worthwhile to present their basic principles. The first of them is known as Dynamic Adaptive Planning (DAP), which aims to build plans that explicitly incorporate provisions for adaptation as changes take place and more knowledge about the system is gained. It includes defining a monitoring system consisting of risk parameters and corresponding thresholds that trigger specific actions (Walker et al., 2001). DAP has been used from traffic safety technology issues to adaptive water management given climatic change scenarios (Rahman et al., 2008).

Dynamic Adaptive Policy Pathways is a DMDU approach that focuses on the dynamic of decision making over time, exploring possible sequences of decisions (adaptation pathways) and their interdependent relations. It also considers that policy actions have a limited lifetime and fail to keep achieving the objectives for which they were designed if conditions change, a moment called adaptation tipping point (ATP) when new actions can be implemented (Haasnoot et al., 2013). The DAPP framework was applied to the long-term water system planning in The Netherlands, even inspiring the Dutch Delta Programme in this region (Haasnoot et al., 2013).

Another key DMDU approach is known as Robust Decision Making (RDM) that focuses on stress testing policies and management strategies over a wide range of future scenarios, trying to find robust policies that perform acceptably over many possible SOW and also trying to identify key conditions that affect policies failure or success (Lempert et al., 2003, 2006). RDM was extensively applied in management of demand and water supply in Colorado River basin (Groves et al., 2013). A relevant extension of RDM is the Many Objective Robust Decision Making (MORDM), an approach that uses many objective evolutionary searches to generate candidate policies, selecting a Pareto set of the ones that perform better in a wide sample of deeply uncertain states of the world. These best set of candidate solutions are tested in broader and more extreme SOW under deep uncertainties in order to find vulnerabilities and performance trade-offs that can aid stakeholders on the decision making process (Kasprzyk et al., 2013).

In this sense, MORDM is a powerful tool since it incorporates different perspectives and objectives into the decision analysis. An extensive effort of research has applied MORDM to coordinate the planning and management of water supply systems in four neighboring cities in North Carolina, USA, considering the context of deep uncertainties (Herman et. al., 2014, 2015; Zeff et al., 2014, 2016; Gold et al., 2019). The framework proposed in this study also incorporates the principles of RDM and MORDM. MORDM has also been applied to the Federal District of Brazil (FDB) water supply system building an adaptive planning framework (Giacomazzo 2020).

FDB has showed recent rapid population growth and irregular occupation, conditions that accentuate socio-economic disparities and inequalities in water and sanitation infrastructure access in the region (Agência Reguladora de Água, Esgoto e Saneamento Básico do Distrito Federal, 2018). The 2016-2018 water crisis in the Federal District highlighted the existence of uncertainties associated with climate and demand growth projections. The water crisis also exposed impacts of some unpredictable conditions in the performance of the water supply system in the Federal District of Brazil (Giacomazzo, 2020), some of them had great potential to be considered as deep uncertainties.

System settings and context

The system settings define which components are important to system representation in a modeling exercise. In this section, we differentiate the system settings for each of the three urban water systems in focus.

Urban Water Supply System

The UWSS representation includes a network of water sources (reservoirs and river uptakes), water treatment plants, water demand sites, links between water sources to allow water transfers and all the characteristics of these elements that define their operation (reservoir capacities, water permits, water losses, water transfer capacities, water treatment capacity and many other inputs). The UWSS can be divided in sets of water sources and their demand sites, defining separate service areas. This approach allows for further analysis, such as identifying possible disparities in socioeconomic conditions and its consequences in adaptation to droughts and other challenging scenarios. Figure 5 shows a schematic representation of a general Urban Water Supply System.

Figure 5
Schematic design of a typical Urban Water Supply System.

Urban Drainage System

The settings of a UDS includes the network of ducts, manholes, detention basins and other infrastructure components that promote infiltration or transfer stormwater to the natural drainage system, which receives the surface runoff volumes. It is recommended that the settings for UDS incorporate sustainable solutions that emphasize the infiltration of stormwater rather than transferring the runoff to downstream rivers.

The operation of the UDS is also related to the land use in the basin, as it determines the amount of runoff produced in every rainfall and therefore the size of the infrastructure planned for the system. Thus, land use maps of the study area may support the definition of parameters that are used in many hydrological models, such as curve number.

The drainage systems also depend on stakeholders and decision makers, as they are responsible for non-structural solutions (usually named housekeeping actions) that are essential for the efficiency of the system. Some examples are the conservation of green areas and floodplains, motivations to control measures at residential units as rain barrels, and financial resources targeting stormwater management and control measures on public areas. These are management actions that stakeholders may consider when planning for urban drainage systems.

Figure 6 shows a schematic representation of a general Urban Drainage System considering the proposed framework. The different city configurations (Figure 6, left) represent some possible urban settings, with different levels of population density, remaining vegetation, and infiltration capacity, for example. As a result, each one of these configurations generates different runoff volumes and pollution loads after a storm event. As an attempt to reduce the impacts on the receiving water bodies (Figure 6, right), several drainage solutions (Figure 6, center) can be adopted at any of these configurations. Therefore, the goal is to find solutions and city configurations (policies) that perform best on reducing the impacts on the water bodies (objectives) across most of the scenarios simulated (SOW).

Figure 6
Schematic design of a typical Urban Drainage System.

Urban Rainwater Harvesting System

The main settings for an URHS are the system water demand, the water catchment area (from the roof top or floors), tanks to store and distribute water, pumping components and water treatment facilities. The system efficiency depends on the capacity of storage tanks that must be sized properly using a water balance simulation. Two basic elements are relevant for the evaluation of the performance of the URHS systems: the water catchment areas and the potable or non-potable water demands in the communities. The description of the systems should also include the existence of any regulation that encourages the implementation of the URHS in the community. Figure 7 shows a schematic representation of a general Urban Rainwater Harvesting System.

Figure 7
Schematic design of a typical Urban Rainwater Harvesting System.

Objectives definition

In this section we describe the objectives selected for each UWS. They represent the expectations and aspirations of the society and community shared vision that emerged from the water system planning and management. These objectives may vary from one system to another and for different communities. It is highly recommended to consult and interview managers, users, communities and other stakeholders in order to define the system objectives. Table 1 presents examples of objectives for each of the three systems analyzed in this study, which should be submitted to validation by decision makers and specialists in a case study.

Table 1
Objectives for different Urban Water Systems.

Urban Water System	Objectives
Urban Water Supply System	- Maximization of reliability, calculated as the percentage of years when the water storage dropped below some percentage of its maximum capacity;
	- Minimization of restriction frequency, represented as the number of years when water restriction measures were applied;
	- Minimization of new Infrastructure Net Present Value (INPV);
	- Minimization of the cost of short term risk mitigation instruments;
	- Minimization of Worst First-Percentile (WFPC) of financial variability caused by drought management actions (restrictions and transfers) among all realizations.
Urban Drainage System	- Minimization of surface runoff volumes and peak flow;
	- Minimization of pollutant loads from the different types of land use;
	- Maximization of avoided costs;
	- Minimization of system costs (infrastructure and maintenance).
Urban Rainwater Harvesting System	- Maximization of the water demand served, taken as a percentage of non-potable demand;
	- Maximization of reliability, represented by the number of days that water demand is fully served;
	- Maximization of rainwater harvested, calculated as the percentage of the total rainwater volume collected that is actually used;
	- Maximization of Net Present Value of the URHS including a balance between costs and benefits (water tariff savings) of the system;
	- Maximization of Net Present Value per volume of consumed rainwater;
	- Maximization of Cost-Benefit Rate.

Identifying Deep Uncertainties (DU)

The DU factors are grouped into vectors, each one randomly paired with the well-characterized uncertainty series in order to build the SOW that represent possible future scenarios. The DU factors values can be chosen randomly using Latin Hypercube Sampling methodology (McKay et al., 1979), considering the predefined lower and upper boundaries of each DU factor’s range. Table 2 presents the set of DU factors for each type of UWS in the present study.

Table 2
DUs defined for each Urban Water System.

Application	Deep Uncertainty Factor
Urban Water Supply System	- New investment discount rate;
	- Water tariff;
	- Effectiveness of water consumption restriction;
	- Time to issue environmental licenses for infrastructure construction;
	- New infrastructure construction costs.
Urban Drainage System	- Stakeholder preferences;
	- Rate of land parceling and overall imperviousness;
	- Land use change;
	- Household use of Low Impact Development (LID) solutions (user cooperation);
	- Effectiveness of selected LID solutions;
	- New investment discount rate.
	- Rate of increase in operational and maintenance costs along the simulation.
Urban Rainwater Harvesting System	- New investment discount rate;
	- Water tariff;
	- Increase rate in operational and maintenance costs along the simulation;
	- Demand (potable or non-potable).

Building States of the World (SOW)

The SOW are comprised of well-characterized and deep uncertainties combined to define future scenarios of system operation. Table 3 presents selected well-characterized uncertainties for each of the UWS in the present study, which form the SOW when associated to DU factors.

Table 3
Well-characterized uncertainties defined for each application.

Application	Well-Characterized Uncertainties
Urban Water Supply System	- Evaporation rates in the reservoirs (water sources);
	- Streamflow (inflows) into the water sources;
	- Future water demand.
Urban Drainage System	- Historical rainfall data.
Urban Rainwater Harvesting System	- Historical rainfall data.

Policy definition to planning and management of the UWS

A policy is a set of measures and actions available to manage the system across the simulation considering the whole spectrum of SOW. Therefore, policies try to manage operational risks and to preserve an acceptable performance of the system during the long-term simulation. Table 4 presents the set of policies evaluated across all SOW according to the water resource system.

Table 4
- Definition of available policies to manage the water system during the simulation.

Application	Policies
Urban Water Supply System	- Short-term actions: water transfers, water consumption restriction, contingency water tariffs, educational campaigns towards rational water use. These are mostly drought mitigation actions.
	- Long-term actions: new infrastructures or infrastructure expansion in the water system and network (reservoirs, pumping, water treatment plants).
Urban Drainage System	- Large scale: long-term planning of land use, LID solutions in public areas.
	- Small scale: individual LID solutions in household (land parcels).
Urban Rainwater Harvesting System	- Small scale: different configurations of the system in terms of water demand and rooftop area.

In the case of Water Supply System, the decision maker has two categories of actions to manage risks, the short and the long-term policies. During the simulation and optimization of the system, short-term actions are implemented to control short-term risks of failure and may contribute to postpone the need of new investments (infrastructures). On the other hand, long-term actions are triggered whenever long-term risks reach unacceptable levels.

In terms of the drainage system, the policies are alternatives of large and small scale LID solutions. They may be implemented separately or in association with each other in order to evaluate the performance of the overall drainage system.

Given the small scale of the rainwater harvesting system, the policies are defined as the configuration of the system itself and its performance is compared among these system configurations. In each policy, the URHS is characterized by the water demand paired to different sizes of rooftop (or floor areas) areas according to reasonable association between the level of demand and the size of the household, representing the correlation between water consumption and average income.

Analysis of DU influence in the system performance: study case for a URHS

The evaluation of system performance considering the presence of deep uncertainty in a set of SOW has been an usual approach among DMDU applications. In general, the studies select the most important objectives of the system which are used as reference to evaluate performance. Then, according to predefined and acceptable performance criteria (minimum or maximum values for the different indicators) we evaluate the policies under a set of SOW to measure their robustness.

The policy robustness can be evaluated in terms of how acceptable a given policy performs over a wide range of different SOW. Specific DU evaluations can also be made by creating more challenging SOW with broader DU multiplying factors and filtering the policies that meet certain performance criteria, as proposed in the satisficing metric methodology applied in Trindade et al. (2019) and Herman et al. (2015). Another interesting analysis consists in using the indicator values obtained in these evaluations to map the space of scenarios (or uncertainties) that fail to meet performance criteria, a method called scenario discovery (Trindade et al., 2019). To illustrate the methodology, we present an analysis of the URHS performance for the Ipameri city in the State of Goiás, Brazil. Figure 8 shows general characteristics of the system. The policies are represented by an association of eight levels of water demand and two roof surface areas generating 16 categories of URHS policies.

Figure 8
Objectives, Policies, Deep uncertainties, climate and other characteristics of the URHS in Ipameri city, State of Goiás, Brazil.

The water supply tariff, operational costs and discount rate were defined as DU factors along the 30-year simulation of the system. The SOW are comprised of 1,000 DU vectors associated with 1,000 daily rainfall series generated using Bootstrap Sampling from historical data. Each DU vector was built applying Latin Hypercube Sampling (LHS) technique in the range defined by the lower and upper bounds in Figure 8. Each URHS configuration (a combination of water demand and roof surface area) was evaluated in the SOW (1,000 scenarios). Once 16 URHS configurations were built, there were 16,000 system evaluations in the framework.

The system water balance of the URHS selected the storage reservoir capacity that resulted in the highest NPV among 18 reservoir capacities. The total costs of URHS included water storage tank, pumping structures, upper water storage reservoir (0.5 m³), filters and accessories. For tanks with volumes greater than 20 m³, the total capacity resulted from the use of two tanks. The operational costs were computed according to the Brazilian Regulation NBR 15527 (Associação Brasileira de Normas Técnicas, 2019) and included energy, water quality sampling and maintenance costs. The economic evaluation considered the water tariff structured by the Water Utility SANEAGO in July 2019 and also used the discount rate based on the Selic rate in November 2021, which was 7.65% per year. (0.6162% per month). Equations for the water balance and economic evaluation are presented in the ^{Appendix A}.

The performance indicators for each configuration are illustrated in Figure 9, which presents a general evaluation of system performance, the tradeoffs among different objectives and the policy robustness. Each line in the graph represents an optimized policy and the vertical axis represents each evaluated objective. The colored lines represent the policies that met the performance criteria defined in this work (SD ≥ 50%; REL ≥ 50%, RH ≥ 30%, NPV ≥ 0, NPVV ≥ 0 and BCR ≥ 1) and the color shades (dark) indicate the reservoir capacity.

Figure 9
Objective tradeoffs and system performance indicators per water demand configuration of URHS in Ipameri.

Since the URHS is not the single source of water in the household, it operates as an alternative source of water supply and levels of SD and REL indicators equal or greater than 50% are acceptable. Among the overall simulation of the 16,000 scenarios of the 16 system configurations that were evaluated, 7,390 reached the required criteria which suggests that this system could be an important strategy for enhancing access to water in urban environment. It is noticeable that in lower water demand systems there is lower performance of the economic indicators, indicating the relevance of public policies based on fundings to motivate the use of URHS in referred communities. Due to the water tariff, households with higher levels of water demand reach better economic indicators for the URHS while technical indicators SD, REL and RH show slight reduction.

Another analysis in DMDU is the sensitivity of the system performance to variability of the DU factors. With that we verify which DU factor has greater impact in the UWS evaluation. Figure 10 illustrates this analysis for the study case where the x axis represents the system objective (SD) and the y axis is the DU factor. The size of the circle represents the accumulation tank capacity, and the colored circles show the scenarios where the objective criteria were accomplished (SD ≥ 50%; REL ≥ 50%, RH ≥ 30%, NPV ≥ 0, NPVV ≥ 0 and BCR ≥ 1).

Figure 10
Sensitivity analysis of DU factors in URHS performance in Ipameri city.

The results in Figure 9 and Figure 10 show a higher NPV for intermediate reservoir storage values and satisfied demand (SD) at around 60% and 80%. The sensitivity analysis indicated the influence of two DU factors in the SD performance criteria, the rate of increase in the water tariff and the rate of increase in the discount rate. The operational costs did not show significant influence on system performance, enhancing the relevance of investment costs in this kind of system (specially in the costs of the reservoir storage).

CONCLUSIONS

Research on deep uncertainty is an emerging theme, as 90% of the publications analyzed are concentrated in the last six years. The bibliometric analysis showed the significant potential for application in several topics in water resources. The concentration of most publications in five countries suggests that the potential for application is still underexplored in other countries and research centers, such as Brazil. In this perspective, the present work offers relevant contribution for decision making under deep uncertainty to be a useful tool to foster scientists, educators, and decision makers in the improvement of water resources management in urban areas.

This work proposed a DMDU framework for adaptive planning and management that is adjustable to urban water systems of three different natures: water supply system, drainage system and rainwater harvesting system. The assessment of each step of the framework, and also between the water systems analyzed, shows that it is possible to adapt the DMDU approach to meet different objectives, according to the water system of interest. Furthermore, the application of DMDU can also be adapted over time, which makes the planning of these systems more flexible, instead of the static and well-defined scenarios that usually drive the traditional decision making process. Hence, the DMDU methodology allows long term planning for these urban water systems even under the many challenges expected due to climatic, economic and social changes in progress at a global level.

Appendix A Water balance method and performance indicators equations.

Water balance method and performance indicators equations:

$V{a}_{\left(t\right)}=\left(\text{P}-\text{di}\right).\text{A}.C_f/1000$(1)

$V{c}_{\left(t\right)}=m\text{í}n\left\{\begin{array}{c}{D}_{\left(t\right)}\\ \theta .V{a}_{\left(t\right)}+V{r}_{\left(t-1\right)}\end{array}\right.$(2)

$V{r}_{\left(t\right)}=m\text{í}n\left\{\begin{array}{c}V{a}_{\left(t\right)}+V{r}_{\left(t-1\right)}-V{c}_{\left(t\right)}\\ \text{K}-\left(1-\theta \right).V{c}_{\left(t\right)}\end{array}\right.$(3)

$SD={{\displaystyle \sum }}_{t=1}^T\frac{V{c}_{\left(t\right)}}{D_{\left(t\right)}}$(4)

$REL=\frac{{{\displaystyle \sum }}_{t=1}^{T}t\left(seV{c}_{\left(t\right)}=D_{\left(t\right)}\right)}{\text{T}}$(5)

$RH=\frac{{{\displaystyle \sum }}_{t=1}^{T}V{c}_{\left(t\right)}}{{{\displaystyle \sum }}_{t=1}^{T}V{a}_{\left(t\right)}}$(6)

Va: available rainwater volume (m³)

P: precipitation in the day (mm)

di: initial disposal (mm)

A: household roof area (m²)

Cf: initial runoff flow coefficient

Vc: rainwater consumed (m³)

Vr: rainwater volume in the reservoir (m³)

D: URHS daily demand (m³)

K: accumulation tank capacity (m³)

ϴ: YAS coefficient (0≤ϴ≤1)

SD: satisfied demand (%)

REL: reliability (%)

RH: rainwater harvested (%)

t: day

T: URHS life time (days)

Economic analysis:

$B_{\left(m\right)}=\left(\left(\left(\text{cf}+\text{Vm}.\text{TA}\right)-\left(cf+\left(Vm-Vc\right).\text{TA}\right)\right)\left(1+pe\right)\right)vTA$(7)

$O{C}_{\left(m\right)}=\left(Em+Emv.Cv+Eq+Eb+Ea+Eav.Inv\right)vOC$(8)

m: month

B: benefit ($)

Vc: volume supplied by URHS in month m (m³)

Vm: volume of water consumed monthly (m³)

TA: water tariff ($/m³)

cf: fixed minimum water tariff ($)

pe: percentage of water tariff corresponding to the sewage (%)

vTA: water tariff variation rate (%)

OC: operating and maintenance expenses ($)

Em: fixed monthly expenses ($)

Emv: variable monthly expenses ($/m³)

Cv: monthly consumption (m³)

Eq: fixed quarterly expenses ($)

Eb: fixed biannual expenses ($)

Ea: fixed annual expenses ($)

Eav: variable annual expenses (%)

Inv: initial investment ($)

vOC: rate of variation in expenses (%)

Economic indicators:

$Re{v}_{\left(m\right)}=B_{\left(m\right)}-O{C}_{\left(\text{m}\right)}$(9)

$BCR=\frac{{{\displaystyle \sum }}_{m=1}^T\frac{Re{v}_{\left(m\right)}}{{\left(1+\left(DR.vDR\right)\right)}^m}}{Inv}$(10)

$NPV={{\displaystyle \sum }}_{m=1}^T\frac{Re{v}_{\left(m\right)}}{{\left(1+\left(DR.vDR\right)\right)}^m}-Inv$(11)

$NPVV=\frac{NPV}{{{\displaystyle \sum }}_{m=1}^{T}V{c}_{\left(m\right)}}$(12)

Rev: revenues ($)

DR: discount rate (% pm)

vDR: rate of variation discount rate (%)

m: month

Inv: initial investment ($)

BCR: cost benefit ratio

NPV: net present value ($)

NPVV: net present value per volume consumed ($/m³)

ACKNOWLEDGEMENTS

The authors thank o Decanato de Pós-Graduação da Universidade de Brasília, o Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), o Instituto Federal de Alagoas (IFAL), o Instituto Federal de Goiás (IFG) e Cornell University

REFERENCES

• Agência Reguladora de Água, Esgoto e Saneamento Básico do Distrito Federal - ADASA. Companhia de Saneamento Ambiental do Distrito Federal - CAESB, Empresa de Assistência Técnica e Extensão Rural do Distrito Federal - EMATER-DF & Secretaria de Estado da Agricultura, Abastecimento e Desenvolvimento Rural - SEAGRI. (2018). Gestão da Crise Hídrica 2016-2018: Experiências do Distrito Federal Brasília: ADASA.
• Agusdinata, B., Marchau, A. W. J., & Walker, W. E. (2006). Exploratory modeling to support a robust policy for implementing intelligent speed adaptation. In 9th International TRAIL Congress Rotterdam.
• Associação Brasileira de Normas Técnicas - ABNT. (2019). NBR 15527: Aproveitamento de água de chuva de coberturas para fins não potáveis - requisitos (2. ed.). Rio de Janeiro: ABNT.
• Bartholomew, E., & Kwakkel, J. H. (2020). On considering robustness in the search phase of Robust Decision Making: a comparison of Many-Objective Robust Decision Making, multi-scenario Many-Objective Robust Decision Making, and Many Objective Robust Optimization. Environmental Modelling & Software, 127, 2020.
• Bonzanigo, L., Rozenberg, J., Felter, G. C., Lempert, R. J., & Reed, P. M. (2018). Building the resilience of WSS utilities to climate change and other threats: a road map (79 p.). Washington: The World Bank.
• Carrera, J., Arroyo, V., Rojas, F., & Mejia, A. (2018). Water security in Latin America: the urban dimension. empirical evidence and policy implications from 26 cities. In World Water Council (Ed.), Global water security: lessons learnt and long-term implications (Water Resources Development and Management) (pp. 217-232). Singapore: Springer.
• Casal-Campos, A., Sadr, S. M. K., Fu, G., & Butler, D. (2018). Reliable, resilient and sustainable urban drainage systems: an analysis of robustness under deep uncertainty. Environmental Science & Technology, 52, 9008-9021.
• Characklis, G., Kirsch, B., Ramsey, J., Dillard, K., & Kelley, C. T. (2006). Developing portfolios of water supply transfers. Water Resources Research, 42(5), 1-14.
• Díez-Herrero, A., & Garrote, J. (2020). Flood risk analysis and assessment, applications and uncertainties: a bibliometric review. Water (Basel), 12(7), 2050. http://dx.doi.org/10.3390/w12072050
  » http://dx.doi.org/10.3390/w12072050
• Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., & Lim, W. M. (2021). How to conduct a bibliometric analysis: an overview and guidelines. Journal of Business Research, 133, 285-296. https://doi.org/10.1016/j.jbusres.2021.04.070
  » https://doi.org/10.1016/j.jbusres.2021.04.070
• Giacomazzo, A. P. (2020). Análise de portifólios de planejamento e gestão do sistema de abastecimento urbano do Distrito Federal no contexto de incertezas profundas (Dissertação de mestrado). Departamento de Engenharia Civil e Ambiental, Faculdade de Tecnologia, Universidade de Brasília, Brasília. Retrieved in 2020, September 1, from http://ptarh.unb.br/dissertacoes-mestrado
  » http://ptarh.unb.br/dissertacoes-mestrado
• Gold, D. F., Reed, P. M., Trindade, B. C., & Characklis, G. W. (2019). Identifying actionable compromises: navigating multi-city robustness conflicts to discover cooperative safe operating spaces for regional water supply portfolios. Water Resources Research, 55(11), 9024-9050.
• Groves, D. G., Fischbach, J. R., Bloom, E., Knopman, D., & Keefe, R. (2013). Adapting to a changing Colorado river: making future water deliveries more reliable through robust management strategies Santa Monica: Rand Corporation.
• Haasnoot, M., Kwakkel, J. H., Walker, W. E., & Maat, J. (2013). Dynamic adaptive policy pathways: a method for crafting robust decisions for a deeply uncertain world. Global Environmental Change, 23(2), 485-498.
• Herman, J. D., Reed, P. M., Zeff, H. B., & Characklis, G. W. (2015). How should robustness be defined for water systems planning under change? Journal of Water Resources Planning and Management, 141(10), 04015012. http://dx.doi.org/10.1061/(ASCE)WR.1943-5452.0000509
  » http://dx.doi.org/10.1061/(ASCE)WR.1943-5452.0000509
• Herman, J. D., Zeff, H. B., Reed, P. M., & Characklis, G. W. (2014). Beyond optimality: multistakeholder robustness tradeoffs for regional water portfolio planning under deep uncertainty. Water Resources Research, 50(10), 7692-7713. http://dx.doi.org/10.1002/2014WR015338
  » http://dx.doi.org/10.1002/2014WR015338
• Kasprzyk, J. R., & Reed, P. (2009). Managing population and drought risks using many-objective water portfolio planning under uncertainty. Water Resources Research, 45, W12401.
• Kasprzyk, J. R., Nataraj, S., Reed, P. M., & Lempert, R. J. (2013). Many objective robust decision making for complex environmental systems undergoing change. Environmental Modelling & Software, 42, http://dx.doi.org/10.1016/j.envsoft.2012.12.007
  » http://dx.doi.org/10.1016/j.envsoft.2012.12.007
• Kwakkel, J. H., Auping, W. L., & Pruyt, E. (2013). Dynamic scenario discovery under deep uncertainty: the future of copper. Technological Forecasting and Social Change, 80(4), 789-800.
• Kwakkel, J. H., Haasnoot, M., & Walker, W. E. (2014). Developing dynamic adaptive policy pathways: a computer-assisted approach for developing adaptive strategies for a deeply uncertain world. Climatic Change, 132(3), 373-386.
• Kwakkel, J. H., Haasnoot, M., & Walker, W. E. (2016). Comparing robust decision-making and dynamic adaptive policy pathways for model-based decision support under deep uncertainty. Environmental Modelling & Software, 86, 168-183.
• Lempert, R. J. (2002). A new decision sciences for complex systems. Proceedings of the National Academy of Sciences of the United States of America, 99(Suppl 3), 7309-7313.
• Lempert, R. J., Groves, D. G., Popper, S. W., & Bankes, S. C. (2006). A general, analytic method for generating robust strategies and narrative scenarios. Management Science, 52(4), iv-638.
• Lempert, R. J., Popper, S., & Bankes, S. (2003). Shaping the next one hundred years: new methods for quantitative, long term policy analysis (209 p.). Santa Mônica: Centro Rand Pardee.
• Marchau, V., Walker, W., Bloemer, P., & Popper, S. (2019). Decision making under deep uncertainty. from theory to practice. USA: Springer.
• Marengo, J. A. (2014). O futuro clima do Brasil. Revista USP, (103), 25-32. http://dx.doi.org/10.11606/issn.2316-9036.v0i103p25-32
  » http://dx.doi.org/10.11606/issn.2316-9036.v0i103p25-32
• McKay, M. D., Beckman, R. J., & Conover, W. J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21(2), 239-245.
• Rahman, A., Walker, W. E., & Marchau, V. (2008). Coping with uncertainties about climate change in infrastructure planning - an adaptive policymaking approach (Final Report). Roterdã, Países Baixos: Ecorys Research and Consulting.
• Trindade, B. C. (2019). Advancing regional water supply portfolio management and cooperative infrastructure investment pathways under deep uncertainty (PhD Thesis). Environmental and Water Resources Systems Program, Cornell University, Ithaca.
• Trindade, B. C., Gold, D. F., Reed, P. M., Zeff, H. B., & Characklis, G. W. (2020). Water pathways: an open source stochastic simulation system for integrated water supply portfolio management and infrastructure investment planning. Environmental Modelling & Software, 132, 104772.
• Trindade, B. C., Reed, P. M., Herman, J. D., Zeff, H. B., & Characklis, G. W. (2017). Reducing regional drought vulnerabilities and multi-city robustness conflicts using many-objective optimization under deep uncertainty. Advances in Water Resources, 104, 195-209.
• Trindade, B., Reed, P., & Characklis, G. (2019). Deeply uncertain pathways: integrated multi-city regional water supply infrastructure investment and portfolio management. Advances in Water Resources, 134, 103442.
• Walker, W. E., Harremoës, P., Rotmans, J., van der Sluijs, J. P., van Asselt, M. B. A., & Janssen, P. (2003). Defining uncertainty: a conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5-17.
• Walker, W. E., Haasnoot, M., & Kwakkel, J. H. (2013a). Adapt or perish: a review of planning approaches for adaptation under deep uncertainty. Sustainability, 5(3), 955-979.
• Walker, W. E., Lempert, R. J., & Kwakkel, J. H. (2013b). Deep uncertainty. In S. I. Gass & M. C. Fu (Eds.), Encyclopedia of operations research and management science (pp. 395-402). New York: Springer.
• Walker, W. E., Rahman, S. A., & Cave, J. (2001). Adaptive policies, policy analysis, and policy-making. European Journal of Operational Research, 128(2), 282-289.
• Watson, A. A., & Kasprzyk, J. R. (2017). Incorporating deeply uncertain factors into the many objective search process. Environmental Modelling & Software, 89, 159-171.
• Yu, Z., Waqas, M., Tabish, M., Tanveer, M., Haq, I. U., & Khan, S. A. R. (2022). Sustainable supply chain management and green technologies: a bibliometric review of literature. Environmental Science and Pollution Research International, 29, 58454-58470. http://dx.doi.org/10.1007/s11356-022-21544-9
  » http://dx.doi.org/10.1007/s11356-022-21544-9
• Zeff, H. B., & Characklis, G. W. (2013). Managing water utility financial risks through third-party index insurance contract. Water Resources Research, 49(8), 4939-4951.
• Zeff, H. B., Herman, J. D., Reed, P. M., & Characklis, G. W. (2016). Cooperative drought adaptation: integrating infrastructure development, conservation, and water transfers into adaptive policy pathways. Water Resources Research, 52(9), 7327-7346.
• Zeff, H., Kasprzyk, J., Herman, J., Reed, P., & Characklis, G. (2014). Navigating financial and supply reliability tradeoffs in regional drought portfolios. Water Resources, 50(6), 4906-4923. http://dx.doi.org/10.1002/2013WR015126
  » http://dx.doi.org/10.1002/2013WR015126
• Zhang, X., Chen, N., Sheng, H., Ip, C., Yang, L., Chen, Y., Sang, Z., Tadesse, T., Lim, T. P. Y., Rajabifard, A., Bueti, C., Zeng, L., Wardlow, B., Wang, S., Tang, S., Xiong, Z., Li, D., & Niyogi, D. (2019). Urban drought challenge to 2030 sustainable development goals. Science of the Total Environment, 693, 133536.
• Zhang, Y., & Chen, Y. (2020). Research trends and areas of focus on the Chinese Loess Plateau: A bibliometric analysis during 1991-2018. Catena, 194, 104798. https://doi.org/10.1016/j.catena.2020.104798
  » https://doi.org/10.1016/j.catena.2020.104798

Edited by

Editor in-Chief: Adilson Pinheiro

Associated Editor: Priscilla Macedo Moura

Publication Dates

• Publication in this collection
  04 Sept 2023
• Date of issue
  2023

History

• Received
  14 Dec 2022
• Reviewed
  20 June 2023
• Accepted
  26 June 2023