UNIVERSITIES IN SMALL BRAZILIAN CITIES: ANALYSIS OF THE IMPACT OF COVID 19 USING VALUE-FOCUSED THINKING

ABSTRACT

This paper aims to support decision-making for the return of classes to determine actions to reduce the problems arising from the pandemic through a multi-methodological approach using Value-focused thinking (VFT) and Rich Picture (RP). The case study includes a Brazilian small university, in a region with social and economic vulnerability. Six stakeholders representing the students, the municipality community and the university were interviewed from which the hierarchy of fundamental objectives and an objectives’ network unfolded in education, health, and community were created. Even though health concerns were shared by all stakeholders, these networks revealed a variety of perspectives not typically discussed in large universities, such as access to information and communication technologies; awareness of biosafety measures; and real estate vacancy. These findings indicate that future research should address the specific needs of small universities in vulnerable cities, particularly those in developing and underdeveloped countries, where universities’ presence fosters the economy.

Keywords:
Education; Soft OR; Problem Structuring

1 INTRODUCTION

Brazil has been experiencing one of the most significant changes in societal habits since the first quarter of 2020. With the onset of local transmission of the Sars-cov-2 virus (COVID-19) in the country’s major metropolises, measures of social exclusion, severe restrictions on people’s movement, and suspension of face-to-face activities were required. These initiatives aim to protect the population’s health and, in particular, to prevent the health system from collapsing due to the high rate of transmissibility, the long average length of hospital stay, and the need to use scarce resources for the most critical diseases, such as the use of intensive care units (ICUs) and respirators (Guimarães et al., 2020; Lee et al., 2020; Ortega & Behague, 2020).

One of the restrictions was the suspension of face-to-face classes in universities. The Ministry of Education authorized the substitution of in-class courses to online learning beginning March 17, 2020, for the duration of the Coronavirus - COVID-19 pandemic situation (Brazil. Ministry of Education, 2020a) and was later changed (Brazil. Ministry of Education, 2020b, 2020c, 2020d, 2020e).

These measures sparked a heated debate about the return of classes throughout Brazil, and because of the wide variety of contexts involved, the challenges take on different formats and characteristics for each of the universities involved. The subject of this study is a university campus in a municipality with a population of less than 15,000 people in Brazil’s semi-arid northeastern region. Approximately 59.1% of Brazilians living in extreme poverty live in the Northeast (Articulação Semiárido Brasileiro, 2020).

This university is the result of higher education expansion funded by the Program to Support Plans for the Restructuring and Expansion of Federal Universities (REUNI). REUNI’s various phases sought to internalize federal public higher education in the country’s most deprived higher education regions: the north and northeast (Camargo & Araújo, 2018; Trombini et al., 2020). Some studies concentrate on rural universities and their specialties, such as attendance patterns, (Byun et al., 2017), rural students’ college success (Manly et al., 2020), impacts of educational reforms (Hu & Hu, 2021) or gender gap in STEM primary choice (Ding et al., 2021).

In addition to increasing access to higher education, universities play an important role in a municipality’s social, cultural, and economic life. Some studies find links between the establishment of universities and the various effects they have on the cities where they are located. Universities, for example, can serve as engines of urban, social, cultural, and economic development in medium and small cities (Baumgartner, 2015b, 2015a; Makagonov et al., 2018; Zborovsky & Ambarova, 2018); the growth of public universities and their implications for the real estate market (Paula & Faria, 2020); as well as the recruitment and retention of graduates in small towns (Imeraj et al., 2018). A survey with data from UNESCO about 15,000 universities in 78 countries between 1950 and 2010, discovered that an increase in the number of universities is positively associated with future GDP per capita growth (Valero & Van Reenen, 2019).

The presence of universities in small and medium-sized cities influences the characteristics of the municipality’s structure. This fact emphasizes the importance of involving social agents such as the population, university community, students, government officials, and economic activity representatives in order to strengthen educational institutions and their relationships with the city (Baumgartner, 2015b, 2015a; Zborovsky & Ambarova, 2018).

This context exhibits characteristics of a wicked problem, such as a diverse set of actors with varying degrees of autonomy and no subordination relationships, uncertainty, and multiple perspectives that direct these stakeholders toward different goals. The difficulty in defining, measuring, or solving the problem is added to the problem’s complexity (Rittel & Webber, 1973; Rosenhead & Mingers, 2001).

Problem structuring methods (PSM) provide a better fit to the traditional approach of Operational Research for this type of problem by focusing on the wicked problem’s conditions (Rosenhead, 1996). As a result, the goal becomes the structuring of the issues and decisions involved, rather than their resolution (Ackoff, 1979; Georgiou, 2008; Jackson, 1982; Rosenhead, 2006; Rosenhead & Mingers, 2001).

Several techniques can be used depending on the intended goal and the characteristics of the problem: Strategic options development and analysis (SODA), for example, is based on the analysis and construction of cognitive maps; Soft systems methodology (SSM) employs modelling techniques based on systems thinking; and Strategic choice approach (SCA) is concerned with managing uncertainty in strategic situations. Workshops with interactive participation are required for the execution of these techniques (Mingers & Rosenhead, 2004; Rosenhead, 1996).

It would be appropriate to combine the use of the Rich Picture (RP) (Checkland, 1972) and Value-Focused Thinking (VFT) (Keeney, 1992), and thus contribute more realistically and effectively to the understanding of the problem. The VFT method was developed in response to the criticism that most decision-making techniques focus on selecting alternatives rather than on how to create them. According to Keeney (1996), alternative-focused thinking is a limiting mode of thought. It is critical to appreciate the values of the stakeholders in order to better understand the conflicts and common interests of the stakeholders and the context. As a result, we sought to include the Rich Picture to provide a broader and more systematic view of the situation, because it can help to open up discussion about individual perceptions about a broad range of issues (Abuabara et al., 2019; Logullo et al., 2022).

Given the socioeconomic impact of universities in small towns and the importance of involving various stakeholders to strengthen the relationship between educational institutions and the host municipality, there is a gap in the involvement of various actors in decision-making regarding the return of face-to-face classes at a university located in a small municipality during the pandemic. In general, the topic is widely debated in light of the university’s interests (Benalcázar et al., 2021; Gherhes et al., 2021).

Through the Value-focused Thinking Problem Structuring Method (VFT) and Rich Picture (RP), the goal of this work was to support decision-making in the return of classes in a university located in the semi-arid northeast in 2021, aiming to determine actions that can reduce the problems arising from the pandemic caused by Sars-CoV-2, promoting the best adjustment to the needs of the stakeholders involved. To accomplish this goal, the research begins with this introduction, then presents a section of the theoretical framework focusing on the theoretical exposition on the VFT and RP, the research methodology is detailed, and finally, the results, discussions, and final considerations are presented.

2 PROBLEM STRUCTURING METHODS AND VALUE-FOCUSED THINKING

In the mid-1960s, methods that are now known as PSM (Problem Structuring Methods) were developed independently. The methods arose in response to criticism of the limitations of traditional operational research, which, in the opinion of those who use it, would be more appropriate for well-structured problems with the consensual formulation (Checkland, 1972, 1981; Rosenhead & Mingers, 2001; Wheeler & Checkland, 2000). These operational research problems would be easily validated in terms of their performance measures, constraints, and the relationships through which the action produces consequences. These methods, however, would be inadequate for many real-world problems involving multiple actors, multiple perspectives, immeasurable and/or conflicting interests, intangible importance, and critical uncertainties (Mingers & Rosenhead, 2004).

The PSMs can thus be defined as a collection of interactive and participatory methods that comprehend the complexities of the context and thus assist the group in understanding the problem, thus responding to the traditional approach of operational research (Franco, 2006; Rosenhead, 1996). Multiple interpretations of the stakeholder group are explored in problem structuring methods to understand its complexity and level of uncertainty in order to achieve a certain level of consensus on objectives (Franco, 2007).

Several PSMs can be characterized according to the objective sought for each problem, including Strategic options development and analysis (SODA), Soft systems methodology (SSM), Strategic choice approach (SCA), Robustness analysis, Viable systems model (VSM), Strategic assumption surfacing and testing (Mingers & Rosenhead, 2004; Rosenhead, 1996) and Value-focused thinking (VFT) (Keeney, 1992).

Concerning the VFT, which was used in this study, Kenney (1992) states that it is a reaction to the limitations of traditional problem-solving methods that focus on alternatives, which characterizes a reactive rather than proactive action, particularly by comparing the alternatives without first articulating the values involved. On the other hand, Keeney (1992) argues that the VFT is constructed in such a way that it assists the decision-maker in focusing on the essential activities that must occur prior to solving the problem, giving the approach greater proactivity when compared to common methods.

2.1 Value-Focused Thinking (VFT)

Value-Focused Thinking (VFT) is an important Problem Structuring Method (PSM), which Parnell et al. (2013) define as “an important philosophy that advocates a more fundamental view of values in decision making in our private and professional lives”. They also acknowledge that VFT has made significant contributions to decision analysis by being used in a variety of applications.

According to Françozo et al. (2019), in the last nine years, the number of papers published on Value-Focused Thinking has increased, with a large portion of the volume of publications shifting from traditional centres to other research centres, such as Brazil and Portugal. The tool’s application to bring the values of decision-makers can be applied in a wide range of subjects, including education and government. These projects involved not only one decision-maker but also a number of stakeholders.

As an example, in the research by Keisler et al. (2014), the VFT was used to pursue the common goals of three community development organizations that included various stakeholders such as the executive director, real estate director, two project managers, and the lead community organizer. As a result, a network of objectives was established to operationalize the generic mission of these organizations, which direct assets and efforts to improve the communities in which they are embedded. In Brazil, Morais et al. (2013) present three different case studies with the application of VFT and communities. The first dealt with water management. In this case, two decision-makers were invited to participate in the application. Engineers, both the process control manager and the energy efficiency manager. The second case involved the strategic planning of information technology in a Public Energy Company with the participation of two chief information officers. The third case involved the disposal of waste from a construction site. This time, a meeting and discussion were held with a specific committee. Finally, the authors recognise the tool’s value, particularly in the context of multiple stakeholders and decision-makers.

Keeney (1996, p. 547) highlights VFT applications in cases involving multiple decision-makers, such as Conflict Management, Inc. (CMI), where the analyst team met “with each of nine principals of CMI to specify and critically examine their objectives”. The company Strategic Decisions Group (SDG) is also worth mentioning, as Keeney “held separate discussions with four partners of SDG to identify the objectives of their compensation system”. In the public sector, the case of an agency responsible for the management of polluted water in the Seattle region of the United States was exposed, in which individual meetings were held with decision-making board members “to elicit values appropriate for Wastewater 2020 Plus, a project to protect water quality through the first half of the 21st century”.

There are several examples in the educational sector, including the research of Françozo et al. (2021), who used the VFT in Brazilian education to structure policies for the inclusion of students with disabilities in Brazilian federal high schools. In Françozo et al (2021), the VFT was used, along with Re-SSM and SODA, to reduce the number of essential objectives to solve the problem, showing the integration of PSM methods.

The value-focused thinking process (VFT), according to Moore (2005), enables the formation of clear connections between values and objectives, thereby identifying better alternatives and decision situations. Values, according to Keeney (1992), are “principles for evaluating the desirability of any possible alternatives and consequences”. In this sense, the VFT aids in the discovery of hidden objectives by emphasizing the significance of stated goals.

The VFT process is divided into several stages: define the values, identify the middle and purpose objectives, structure the network of objectives, and create and select alternatives. Interviewing decision-makers and stakeholders involved with the wicked problem is the first step in identifying values (Keeney, 1996). This paper will not generate and evaluate alternatives because it does not seek to solve the problem but rather to aid in the comprehension and organization of the scenario presented.

2.2 Rich Picture

The Rich picture is a key component of Checkland’s Soft Systems Methodology (SSM) (1981), which was first described in Checkland (1972). The seven-stage methodology is applied to realworld problem situations, with the Rich Picture described in stage 2 as a non-systemic analysis of what exists at the time in the problem situation. Patching (1990) considers a Rich Picture to be a useful alternative to a textual description of the problem situation rather than an SSM requirement and uses the phrase as a label for a diagram type.

The Rich Picture is a free-form drawing created during an interview that allows those involved to understand the problem rather than just read about it. It can also quickly represent the materialization of a problem situation so that people with different perspectives can understand the systems involved. Furthermore, it rationally organizes the visually provided statements, identifying transformations and recording contexts, actors, and other information required to improve the situation (Checkland & Poulter, 2020; Georgiou, 2012; Patching, 1990).

Armson (2011) establishes the following premises for the development of Rich Picture: (i) do not structure the Rich picture; (ii) do not use too many words; (iii) do not exclude relevant observations about culture, emotions, and values; (iv) include multiple points of view; (v) include a self-portrait; and (vi) include a title and a date.

3 RESEARCH METHOD

From methodology, the research classification and stages were developed. The research framework can be understood in the theoretical context presented, as shown in Figure 1.

Figure 1
Research framework.

The combination of the Rich Picture (Checkland, 1972) and Value-Focused Thinking (VFT) (Keeney, 1992) was appropriate because a diagram of the issue would help to clarify the situation around the decision (Abuabara et al., 2019; Logullo et al., 2022). Finally, there is no “right” or “wrong” solution because any designed solution encounters the limitations of a specific agent or stakeholder. The tools chosen are intended to assist decision-makers in narrowing the important points to consider minimizing societal impacts. Figure 2 depicts the research steps that will be discussed in greater detail in subsequent subsections.

Figure 2
Research steps.

3.1 Problem context

The context depicts the earlier knowledge the authors had about the issue before to the interviews.

3.1.1 Definition of the problem and objectives expected to be achieved

To correctly identify the various stakeholders affected by the problem, it was first necessary to clearly define the problem. Four (4) meetings were held among the analysts, during which the objectives and expected gains were discussed after employing the VFT method. The context of the problem was investigated by consulting official documents of the institution, such as reports, ordinances, resolutions, and analysis of the municipality and region via the IBGE (Brazilian Institute of Geography and Statistics) website.

As the problem deepened, the team of analysts identified 3 objectives to be explored and validated with stakeholders: (i) Maximize the performance of the teaching and learning process; (ii) Minimize health impacts, and (iii) Minimize impacts on the local economy. Based on the objectives, it was possible to identify which groups of agents should be consulted and interviewed.

3.1.2 Definition of stakeholders involved

The university plays an important role in the routine of the city that houses its campus, which is located far from large urban centres and is economically dependent on university students. In this regard, the first group of agents consists of city residents and merchants (1) - Community members who not only have a direct interest in the university’s decision but are also directly impacted by it, despite the fact that they do not have the right to participate in the decision-making process.

The second group of agents refers to the students themselves (2) - Students, who are also directly impacted by the decision. Some problems may arise regarding online education due to different income levels and their difficulties regarding digital inclusion. On the other hand, the return to local classes would expose these students to the risks of contamination by COVID, and the consequent risk of contaminating the other inhabitants and family members of their households. To represent this group, the campus representative of the Student Union was chosen.

The third type of agent is political actors (3) - Community. Because of the city’s small size, the return of students to local classes could have a significant impact on the municipal health infrastructure. Not to mention the increased demand for public transportation, which, if social distancing is maintained, would necessitate an increase in available buses, exceeding the municipality’s offered capacity. A city hall official was chosen to represent this group.

The university’s staff (4) - University would be the fourth group of stakeholders. On the one hand, online teaching makes some activities, such as laboratory classes, difficult to perform; on the other hand, returning to local classes would expose this group to contamination, where many make up for risk groups. A member of a committee formed to deal with the pandemic situation on campus and advise campus management was chosen to represent this group. This committee includes members from various sectors (library, laboratory, student assistance, and teachers, for example).

The fifth group of agents are the leaders of the university campus itself (5) - University decision-makers at the university. The campus administration and the academic coordinator were chosen to represent this group, as they are directly responsible for monitoring teaching, research, and extension activities. The problem at hand involves multiple agents with opposing viewpoints. To ensure that all impacted stakeholders were represented, they were divided into three categories: University (Committee Representative, Academic Coordinator, and Campus Direction), Students (Student Union), and Community (Representative of Residents, and City Hall).

The selection of stakeholders is critical for the VFT because the selection of agents with power is important in the definition of objectives. The Eden & Ackermann Matrix (1998) was built and the agents were classified to better understand the relationship between interest and power in solving the problem. Although not all stakeholders are players, that is, with high interest and power, all are important for understanding the problem. It is observed that, despite being influenced by the decision, not all of the agents presented thus far participate in it. Figure 3 depicts the stakeholder’s interest and power matrix.

Figure 3
Stakeholder’s matrix of interest and power. Source: Authors (2022) based on Eden and Ackermann (1998).

3.2 Understanding the problem

Represents the step of direct contact with the agents involved in the problem, so the context of the problem was improved to a view based on the interviewees’ arguments, therefore more realistic.

3.2.1 Planning and Conducting Interviews

Once the key stakeholders were identified, a script for conducting interviews was created. Evidently, new questions were added based on the responses of each interviewee in order to maximize the collection of information from each source. The fundamental questions were as follows:

1. What are the expected consequences if classes do not return face-to-face in 2021?

2. What advantages and disadvantages, in your opinion, will the return of face-to-face/hybrid classes bring to the various groups involved?

3. What would be the most difficult challenges in resuming face-to-face classes?

4. What are the top priorities for resuming face-to-face classes?

5. What are the expected consequences if classes return to face-to-face classes in 2021?

The final point of importance was the attention paid to the duration of the interview; because everyone had busy schedule, the interviews were limited to 1 (one) hour in length. It is worth noting that all interviews were conducted remotely via web conferencing applications, with at least two analysts present at all times to allow for the subsequent exchange and comparison of annotations.

3.2.2 Review of initial assumptions

Following the interviews, the analysts discovered that some of the initial premises required revision. As a result, all analysts gained a better understanding of the problem.

3.2.3 Construction of the Rich Picture

According to Armson’s premises (2011), the analysts began to construct a Rich Picture using the information gleaned from the interviews. One of the analysts started elaborating the Rich Picture’s initial outline, which was later built collaboratively with the other analysts. It is important to note that there are no correct Rich Pictures because each observer or author will have a unique perspective and interpretation of the analyzed situation.

3.2.4 Validation of the Rich Picture with stakeholders

Following preparation, the Rich Picture was presented to the interviewees, who were asked if their concerns were adequately understood and represented. As a result, it was possible to ensure that the information gathered during the interviews was correctly understood and translated for analysis. Four of the six stakeholders interviewed validated the findings.

3.3 VFT - Value-focused thinking

Pursuing validation of the information obtained from stakeholders through interviews and the Rich Picture, the VFT was carried out in accordance with Keeney’s guidelines (1992). It was possible to define the fundamental objectives and half-end objectives during two analyst meetings in order to finally connect them.

3.3.1 Hierarchy and Network of Objectives

The fundamental objective hierarchy is created by categorizing the fundamental objectives into levels. According to Keeney (1992), having two or more lower levels ensures that the goal is dismembered for greater understanding. It is important to note that an objective consists of three parts: a decision context, an object, and a direction (maximize or minimize). The analysts decided to keep this structure throughout the hierarchy. Based on the values of the interviewees, it became clear which objectives characterized the fundamental objectives established at the start of the study.

The primary objectives and means were extracted from each conducted interview. This means that its values were derived from stakeholders and explained by its objectives. The following criteria were used for the formulation of the objective-means-end network: each objective was evaluated, and similar objectives were grouped into a single objective, with a nomenclature that represents it. This ensures that the objective description does not have the characteristics of a specific stakeholder, even if it represents common goals. Following that, the objectives that emerged specifically in some stakeholders and represented objectives of a certain profile were included, so that the network of means-end objectives broadly represented all decision-makers involved.

Once the objectives are listed, they are linked, using the question method: “Why is this goal important?”. If the answer to this question is to achieve another goal, then it receives an arrow. The analysts chose not to follow the objective structure instructed by Keeney (1992) regarding the direction (maximize or minimize) and use a verb in the infinitive form. This decision was made in order to make the network more didactic, and more easily built since it has a larger number of objectives and a limited construction space. The network is important to understand what gaps exist and complete them with objectives that will help to understand the problem.

4 PROBLEM CONTEXT

The concerned university is located in the semi-arid northeast. Its main campus is in the state’s second-largest city in terms of population. There are also three additional campuses outside of the city. The university has approximately 10,000 students and offers 45 undergraduate courses as well as 30 graduate courses. This university has joined the Federal Educational Institutions Restructuring and Expansion Program (REUNI). The program’s main goal was to increase access and retention in higher education by increasing vacancies in undergraduate courses, promoting pedagogical innovations, and combating evasion, in order to reduce social inequalities in the country (Brazil, 2007).

The municipality is expected to have 11,705 residents by 2020, with formal workers earning an average monthly salary of around 1.9 minimum wages (approximately 254 dollars). External sources, such as the federal government, accounted for approximately 93.5% of the city’s revenue (IBGE, 2020). The expenses incurred for the Campus’s operation, including financial assistance, job site location, and general services, added up to approximately $6 million during the 2019/2020/2021 fiscal years (according to the federal government’s personnel administration system). A value greater than what the municipality collected in taxes during the same period, which added up to $4 million and 400 thousand reais, according to the municipality’s Annual Budget Law. These figures show how small the municipality is in relation to Brazil, how important the University is in stimulating and fostering local activity, and the importance of carrying out this type of study in municipalities of different population sizes.

The water supply of the municipality is dependent on pipelines that transport water from dams to the municipality. As a result, the supply is frequently disrupted for corrective and preventive maintenance. Furthermore, only 38.3% of households in the municipality have adequate sewage. With the lack of equipment and basic infrastructure to deal with public health during a pandemic, this factor adds to the number of hospitalization beds in the Unified Health System (SUS - Sistema Único de Saúde), which totals 28 units (IBGE, 2020).

The municipality’s public safety is shared with other municipalities, with limited hours for attendance and occurrence registration. In terms of transportation, there are no lines connecting most students’ home cities to the municipality. This is compounded by the city’s lack of public transportation. University students rely on a single round-trip bus, their own transportation, or a motorcycle taxi.

The campus has 1629 students, 13% of whom are from the municipality. The others are from Ceará and Rio Grande do Norte, respectively. When classes were normally held, the weekly commuting movement of students, technicians, and teachers resulted in a population increase of 14%-15%. The university began in June 2020 with a 6-week online semester that students could choose to enrol in. According to internal information system reports, 39% of students on campus did not enrol this semester for a variety of reasons, including a lack of infrastructure such as internet, access to technology and communication devices, and appropriate conditions for home study. These characteristics compound the economic and financial vulnerability that some students face. Subsidies are essential for students to remain at university, and they also represent an increase in the municipality’s economy.

Figure 4 was created to ensure that the objectives defined in the problem identification stage (3.1.1) were covered by the various stakeholders chosen. Each circle represents one of the main objectives. Intersections occur when a stakeholder has multiple interests or addresses multiple goals.

Figure 4
Venn diagram of the correlation between objectives and agents.

Figure 4 shows that, according to the analysts’ preliminary assessment, all stakeholders are concerned with “Minimizing health impacts” in addition to one of the other objectives. While City Hall and the population are concerned about the effects on the local economy, the Student Union, the Covid 19 Coordination Committee, the Academic Coordinator, and campus management are concerned about the learning process’s performance aspects. Finally, none of the stakeholders chosen is concerned with all three objectives at the same time.

5 UNDERSTANDING OF THE PROBLEM

The interviews revealed that the Venn Diagram of the correlation between objectives and agents needed to be reviewed. The main correction addressed the public’s concern about the effects on the local economy. What could be seen was that the federal government’s creation of emergency aid replaced the financial resources brought by the students, and thus there was no impact on the local economy.

According to the population, as expressed by one of the stakeholders, the absence of students reduced the problems and disturbances caused in the city due to the consumption of alcoholic beverages and drugs. Because the municipality is small, information of this nature spreads quickly. As a result, the population’s primary concern shifted away from local economic issues and toward health-related issues. Figure 5 depicts a new correlation between objectives and stakeholders as a result of this information.

Figure 5
Venn diagram of the correlation between objectives and agents revised.

As presented in the methodology, returning to classes during the pandemic is a problem that involves many stakeholders. As a result, the Rich Picture is an essential component for comprehending this context. The team of analysts was able to elaborate the Rich Picture that depicts the transformations, actors, and information required for greater clarity of the problem’s context through six interviews with stakeholders (Checkland & Poulter, 2020; Georgiou, 2012; Patching, 1990).

The need to vaccinate the population was regarded as critical by all parties involved for a safe return to local classes. The National Plan for the Operationalization of the Covid-19 Vaccine was launched on December 16, 2020, with the goal of vaccinating priority groups in the first half of 2021. (Brasil, 2020). Given this timeline, university planning is still incomplete, as teachers are not among the risk groups, and no vaccine has yet been approved by ANVISA - the Brazilian National Health Surveillance Agency.

The plot of the figure depicts the elements that characterize semi-arid inner cities, such as high temperatures and small homes that house large families. The student is at the centre of the debate between online and face-to-face instruction, weighing the key differences between the two modes.

Even without the existence of local classes in the municipality, ‘emergency aid’ was a determining point for the municipality to have a good income in 2020, moving the local economy. It was also noted that new businesses have been opened. However, it is perceived that there is apprehension among the population and small business owners about the negative consequences of the end of aid, as well as the non-return of local classes.

The presence of the university in the municipality is associated with increased crime and drug use, in addition to the positive aspects such as the growth and development of the municipality. According to information obtained from stakeholders present in the research, crime indicators were reduced in the meantime during the pandemic, with the occurrence of online classes and, as a result, the suspension of activities on campus.

Some advantages of face-to-face classes stand out, such as access to libraries and laboratories. However, in order for classes to take place safely, interviewees were emphatic about the need for protocols, hygiene equipment, the availability of 70% alcohol, totems, face shields, reducing the available seats in each class due to the new social distancing requirements, and other resources that are in accordance with the biosecurity booklet to be adopted by the campus (based on the instructions of the federal government). In this regard, campus administration considers how to reallocate capital costing resources saved during the online semester. Figure 6 was created using these elements.

Figure 6
Rich Picture.

The Rich Picture was validated by four stakeholders after the analysts created it. It was discovered that stakeholders in general understood the Rich Picture. However, the information that characterizes the municipality’s local economy was not well understood. It was perceived that the problem of returning to university classes is viewed solely as an internal challenge for the institution and its participants. The importance of the VFT application is perceived, considering the various stakeholders also associated with the community. As a result, a more comprehensive view of the problem at hand will be possible.

6 VFT APPLICATION, RESULTS, AND DISCUSSIONS

The VFT became relevant because it assisted in understanding the values of various stakeholders and, as a result, elaborating the fundamental objectives of the problem. It was understood as a strategic objective “To minimize the negative effects of the pandemic in the return to classes of a university in the semi-arid northeast”. This was divided into three fundamental objectives, each related to reality: education, health, and community economics. Without the VFT, it would be more difficult for the agents involved to understand how their objectives and values interact with those of the other agents, which was exacerbated by the University’s lack of approximation with the local community. Table 1 shows the hierarchy of built-in objectives.

Table 1
VFT - Hierarchy of objectives.

Strategic objective: Minimize the negative effects of the pandemic on back-to-school at a university in the semi-arid Northeast
1. Maximize the performance of the teaching and learning process
	1.1 Maximize the number of active students
		1.1.1 Minimize the amount of locks
		1.1.2 Maximize the number of students enrolled in the period
	1.2 Minimize the difference in access to information and communication technology
		1.2.1 Maximize the number of students with access to remote learning technologies
		1.2.2 Minimize the impacts of low technological quality on student learning
	1.3 Maximize the quality of the teaching and learning process
		1.3.1 Maximize the result of courses in ENADE
		1.3.2 Maximize the alignment between skills required by the labor market and academic training
		1.3.3 Maximize the use of technologies in teaching practice
2. Minimize damage to health
	2.1 Minimize damage to the health of the academic community
		2.1.1 Minimize contamination rate
		2.1.2 Maximize the dissemination of biosafety protocols
			2.1.2.1 Maximize the availability of hygiene and cleaning materials
			2.1.2.2 Maximize awareness of biosafety protocols
	2.2 Minimize damage to the health of the community
		2.2.1 Minimize contamination rate
		2.2.2 Minimize the occupancy rate of hospital beds
		2.2.3 Maximize dissemination and awareness of preventive measures
3. Minimize impacts on the local economy
	3.1 Minimize the number of closed establishments
	3.2 Minimize economic impacts on university-related segments
		3.2.1 Minimize real estate vacancy rate
		3.2.2 Minimize the reduction in revenue in commercial and service establishments

Concerning the goal of “Maximizing the performance of the teaching and learning process”, it was perceived that interviewees from the university and students expressed concern about the effects on dropout. The students enrolled in the period are those who chose to take the online course rather than perform and wait for the face-to-face classes to resume. Even if the student is enrolled in the period, he or she has the option of cancelling one or more subjects, or the entire semester. Then, with the goal in mind, one can define active students as those who enrolled in subjects and attended until the end of the semester.

Because many students do not have access to quality information and communication technologies (ICTs), and those who do sometimes acquire low-quality equipment that does not allow for an online class or access to the institution’s information systems, access to these technologies may interfere with remote teaching. Furthermore, failure to complete field activities, group activities, and laboratories may jeopardize students’ performance in the ENADE (National Student Performance Exam for Undergraduate Courses), the main evaluation of higher education universities in Brazil, as well as in the labour market. The impact of technology on teaching practice was also highlighted, given that some teachers were unfamiliar with some new technologies and were experiencing them for the first time during the pandemic.

All stakeholders were concerned about their health. However, it was discovered that the efficacy of biosafety measures and the reduction of contamination rates in the university are dependent on the dissemination of protocols, as well as their awareness and material availability. The number of hospitals and ICU beds in the municipality exacerbates this concern. Hygiene and cleaning materials even involve basic resources, such as water, that are difficult to obtain in the region, as well as the university’s financial commitment for this purpose. Furthermore, biosecurity protocols necessitate the use of more ventilated rooms and structures to measure the temperature of everyone entering college buildings.

Because the presence of the university stimulates the local economy, the city’s commerce was affected by the suspension of the local class, and to a lesser extent in 2020, due to the provision of emergency assistance by the federal government (Brazil, 2020). Due to the decrease in the value of emergency aid and its impending closure, this was becoming a concern for the year 2021. During the interviews, it was mentioned that the region’s trade profile had changed during the time period studied, with fewer enterprises aimed at the university public and more trade in basic services. The cancellation of rents and inn reservations has reduced the rate of real estate occupancy, which has been impacted by the suspension of local classes. It is possible to create a network of half-end objectives after constructing the hierarchy of objectives (Figure 7).

Figure 7
VFT - Network of objectives.

It is perceived that the community’s situation is more related to population health because the prevention of coronavirus requires economic measures, such as social distancing. This is not so clearly reflected in education. However, another relevant half-end goal can be seen in the network: compliance with biosafety measures, which complicates the problem even further.

7 FINAL CONSIDERATIONS

The pandemic caused by Sars-Cov-19 affected all economic activities in Brazil and around the world, without exception. One of them was higher education, which had classes suspended and later shifted to an online format until the end of 2021. Brazil is a large country, which exacerbates existing inequalities in the population, such as in the Northeast region, where approximately 60% of the population lives in extreme poverty (Articulação Semiárido Brasileiro, 2020). Considering a semi-arid region of this state, because particularities should be considered when dealing with difficulties, and that the university moves monetary values superior to the municipality, this work addressed the following question: “How Value-focused Thinking can support back-to-school decision-making in a campus in a small Brazilian city in 2021, aiming to minimize the impacts caused by the COVID-19 pandemic and promoting the best adjustment to the needs of stakeholders (University, Community and City management)?”

In this work, Value-Focused Thinking was used to better understand a complex problem: the decision to return to school during a pandemic, which does not have a clear and unanimous solution. It was necessary to first define the agents involved: city residents and merchants, students, political actors, university staff, and university campus leaders, and then to understand their values alongside them, which are critical for the construction of better alternatives. Despite the fact that the interviews were conducted individually, the use of VFT was critical in highlighting the common values for solving the problem, showing the various angles of the problem, and making its dimension clear to stakeholders.

Six stakeholders were interviewed, allowing for the observation of conflicting and common interests for the study’s construction. The analysts’ initial understanding of the problem enabled the elaboration of some premises to initiate an understanding of the problem and provide an adequate and diverse range of stakeholders. During the interviews, it was possible to challenge some established premises while also adding new facts to the research, such as the importance of emergency assistance for the municipality’s financial health and, as a result, an adjunct view of the importance of returning to local classes in the municipality.

The combination of VFT and Rich Picture is one of the work’s highlights. RP helped in structuring the problem with VFT, considering that a diagrammatic view helps in understanding the context involved in decision making. The Rich Picture was validated by four stakeholders who claimed to recognize most of the situations presented. Stakeholders were surprised by the completeness and complexity of the image generated, given the contribution of several participants who did not communicate with one another. The case study aided stakeholders in determining the potential impact of the decision in the real world and in identifying alternative decision options. It is important to note that the Rich Picture is not a finished and frozen work, but rather represents the current state of this complex problem, which is constantly changing.

During the case, the feedback from the university campus management team had a significant practical impact. The issues raised in the interviews were deemed critical. Because health was the central point for all stakeholders, the interviewees realized the scope of the stakeholders involved and that they should collaborate in favour of an integrated solution to this problem. This is a practical outcome that demonstrates the technique’s utility and simplicity.

As a limitation, we highlight the fact that a citizen represented the population’s viewpoint. Despite the possibility of disagreement, it is already a starting point to highlight the opinion of this stakeholder, who would otherwise be ignored. Another limitation of this study is the lack of investigation into potential problem solutions as well as the development of monitoring indicators.

The measures of social isolation interfered with the methodology because they made face-to-face workshops impossible, in addition to being a fundamental protection measure of the problem situation faced. Efforts have been made to mitigate these effects, such as the use of messaging tools and Google Meet. This tool eliminated the need for additional costs by allowing interviews to be recorded and thus repeated to identify potential flaws in interpretation.

The opportunity to analyze the problem is observed for future studies using the Soft Systems Methodology (SSM). Value-Focused Thinking (VFT) also provides tools for the continuation of this study, such as the development of attributes for the objectives and, later, the development of alternatives. A multimethodological model could also be used to apply other Solution Structuring Method tools. In addition to these suggestions, tools such as MIRO and Whiteboard, which enable the emulation of virtual co-creation workshops, can be used to improve the methodology.

Because of the education expansion in Brazil, many federal educational institutions are located in small municipalities and in economically and socially vulnerable areas. As a result, this work can help other educational institutions, communities, and school administrators who are facing similar decision-making challenges as a result of the pandemic.

The presented problem also provides opportunities to investigate the community impact of the university’s establishment. What can be analyzed using socioeconomic indicators such as population growth, Gross Domestic Product in the municipality, or tax collection, for example. As a result, a picture before and after the university’s presence in the small town can be established.

It is worth noting that municipal elections were held in 2020, so the pandemic’s management changed at the start of 2021, a cycle that also includes the completion of this work’s analyses. According to news reports and press releases published by the city and the university campus, there was an agreement between the campus management and the city in the second quarter of 2021 to enrich the debate; plan vaccination of higher education workers who are part of the federal government’s National Immunization Plan as a priority group; and ensure that municipal decrees allow greater biosecurity for the population, based on the epidemiological situation diagnosed in the city.

The number of cases and deaths increased significantly as a result of this shift. From the start of the pandemic to December 2020, 188 cases and 6 deaths were reported, while from January to May 2021, 1318 new cases and 30 new deaths were reported. A 700% increase in cases and a 500% increase in deaths (Brazil, 2021). It is worth noting that this study was completed in 2020, before the pandemic ended and vaccinations began in the country. It was decided at a meeting of the university’s board of directors that classes would be held online during the first half of 2021.

References

• ABUABARA L, PAUCAR-CACERES A & BURROWES-CROMWELL, T. 2019. Consumers’ values and behaviour in the Brazilian coffee-in-capsules market: promoting circular economy. International Journal of Production Research, 57(23), 7269-7288. Available at: https://doi.org/10.1080/00207543.2019.1629664
  » https://doi.org/https://doi.org/10.1080/00207543.2019.1629664
• ACKOFF RL. 1979. The future of operational research is past. Journal of the Operational Research Society, 30(2), 93-104. Available at: https://doi.org/10.1057/jors.1979.22
  » https://doi.org/https://doi.org/10.1057/jors.1979.22
• ARMSON, R. 2011. Growing Wings on the Way: Systems Thinking for Messy Situations. Triarchy Press Ltd.
• ARTICULAÇÃO SEMIÁRIDO BRASILEIRO. 2020. Semiárido. É No Semiárido Que a Vida Pulsa! Available at: https://www.asabrasil.org.br/semiarido
  » https://www.asabrasil.org.br/semiarido
• BAUMGARTNER WH. 2015a. Cidades Universitárias, Cidades Médias, Cidades Pequenas: Análises Sobre o Processo de Instalação de Novos Campi Universitários. Espaço Aberto, 5(1), 73-93. Available at: https://doi.org/10.36403/espacoaberto.2015.2525
  » https://doi.org/https://doi.org/10.36403/espacoaberto.2015.2525
• BAUMGARTNER WH. 2015b. Universidades públicas como agentes de desenvolvimento urbano e regional de cidades médias e pequenas: uma discussão teórica, metodológica e empírica. GeoTextos, 11(1), 91-112. Available at: https://doi.org/10.9771/1984-5537geo.v11i1.12013
  » https://doi.org/https://doi.org/10.9771/1984-5537geo.v11i1.12013
• BENALCÁZAR ME, BARONA L, VALDIVIESO ÁL, VIMOS VH, VELASTEGUI D & SANTACRUZ CJ. 2021. Educational Impact on Ecuadorian University Students Due to the COVID-19 Context. Education Sciences, 12(1): 17.
• BRASIL. 2019. Presidente da República e ministro Eduardo Pazuello apresentam plano de vacinação contra a Covid-19: “Todas as Vacinas Produzidas No Brasil Terão Prioridade Do SUS.” Available at: https://www.gov.br/saude/pt-br/assuntos/noticias/presidente-da-republica-e-ministro-eduardo-pazuello-apresentam-plano-de-vacinacao-contra-a-covid-19-201ctodas-as-vacinas-produzidas-no-brasil-terao-prioridade-do-sus201d
  » https://www.gov.br/saude/pt-br/assuntos/noticias/presidente-da-republica-e-ministro-eduardo-pazuello-apresentam-plano-de-vacinacao-contra-a-covid-19-201ctodas-as-vacinas-produzidas-no-brasil-terao-prioridade-do-sus201d
• BRAZIL. 2007. Decreto n^o 6.096, de 24 de abril de 2007. Institui o Programa de Apoio a Planos de Reestruturação e Expansão Das Universidades Federais - REUNI. Available at: http://www.planalto.gov.br/ccivil_03_ato2007-2010/2007/decreto/d6096.htm
  » http://www.planalto.gov.br/ccivil_03_ato2007-2010/2007/decreto/d6096.htm
• BRAZIL. 2020.Lei n^o 13.982, de 2 de abril de 2020. Altera a Lei N^o 8.742, de 7 de Dezembro de 1993, Para Dispor Sobre Parâmetros Adicionais de Caracterização Da Situação de Vulnerabilidade Social Para Fins de Elegibilidade Ao Benefício de Prestação Continuada (BPC), e Estabelece Medidas Excepcionais de Pr. Available at: http://www.planalto.gov.br/CCIVIL_03/_Ato2019-2022/2020/Lei/L13982.htm
  » http://www.planalto.gov.br/CCIVIL_03/_Ato2019-2022/2020/Lei/L13982.htm
• BRAZIL. MINISTRY OF EDUCATION. 2020a. Portaria n^o 343 de 17 de Março de 2020. Dispõe Sobre a Substituição Das Aulas Presenciais Por Aulas Em Meios Digitais Enquanto Durar a Situação de Pandemia Do Novo Coronavírus - COVID-19. Available at: http://www.planalto.gov.br/CCIVIL_03/Portaria/PRT/Portariano343-20-mec.htm
  » http://www.planalto.gov.br/CCIVIL_03/Portaria/PRT/Portariano343-20-mec.htm
• BRAZIL. MINISTRY OF EDUCATION. 2020b. Portaria n^o 345 de 19 de Março de 2020. Altera a Portaria MEC N^o 343, de 17 de Março de 2020. Available at: https://pesquisa.in.gov.br/imprensa/jsp/visualiza/index.jsp?jornal=603&pagina=1&data=19/03/2020&totalArquivos=1
  » https://pesquisa.in.gov.br/imprensa/jsp/visualiza/index.jsp?jornal=603&pagina=1&data=19/03/2020&totalArquivos=1
• BRAZIL. MINISTRY OF EDUCATION. 2020c. Portaria n^o 395 de 15 de Abril de 202. Prorroga o Prazo Previsto No § 1^o Do Art. 1 ^o Da Portaria N^o343, de 17 de Março de 2020. Available at: https://www.in.gov.br/en/web/dou/-/portaria-n-395-de-15-de-abril-de-2020-252725131
  » https://www.in.gov.br/en/web/dou/-/portaria-n-395-de-15-de-abril-de-2020-252725131
• BRAZIL. MINISTRY OF EDUCATION. 2020d. Portaria n^o 473 de 12 de Maio de 2020. Prorroga o Prazo Previsto No § 1^o Do Art. 1 ^o Da Portaria N^o 343, de 17 de Março de 2020. Available at: https://www.in.gov.br/en/web/dou/-/portaria-n-473-de-12-de-maio-de-2020-256531507?i
  » https://www.in.gov.br/en/web/dou/-/portaria-n-473-de-12-de-maio-de-2020-256531507?i
• BRAZIL. MINISTRY OF EDUCATION. 2020e. Portaria n^o 544 de 16 de Junho de 2020. Dispõe Sobre a Substituição Das Aulas Presenciais Por Aulas Em Meios Digitais, Enquanto Durar a Situação de Pandemia Do Novo Coronavírus - Covid-19, e Revoga as Portarias MEC N^o 343, de 17 de Março de 2020, N^o 345, de 19 de Março de 2020, e N^o 473, de 12. Available at: https://www.in.gov.br/en/web/dou/-/portaria-n-544-de-16-de-junho-de-2020-261924872
  » https://www.in.gov.br/en/web/dou/-/portaria-n-544-de-16-de-junho-de-2020-261924872
• BRAZIL. MINISTRY OF HEALTH. 2021. Painel Coronavírus Brasil. Available at: https://covid.saude.gov.br/
  » https://covid.saude.gov.br/
• BYUN S, YONG MEECE JL & AGGER CA. 2017. Predictors of College Attendance Patterns of Rural Youth. Research in Higher Education, 58(8), 817-842. Available at: https://doi.org/10.1007/s11162-017-9449-z
  » https://doi.org/https://doi.org/10.1007/s11162-017-9449-z
• CAMARGO AMM DE & ARAÚJO IM. 2018. Expansão e interiorização das universidades federais no período de 2003 a 2014: perspectivas governamentais em debate. Acta Scientiarum. Education, 40(1), 37659. Available at: https://doi.org/10.4025/actascieduc.v40i1.37659
  » https://doi.org/https://doi.org/10.4025/actascieduc.v40i1.37659
• CHECKLAND, P. 1972. Towards a systems-based methodology for real-world problem-solving. Journal of Applied Systems Engineering, 3(2), 87-116.
• CHECKLAND, P. 1981. Systems thinking, systems practice. John Wiley & Sons.
• CHECKLAND P & POULTER, J. 2020. Soft systems methodology. In: Systems Approaches to Making Change: A Practical Guide (pp. 201-253). Springer.
• DING Y, LI W, LI X, WU Y, YANG J & YE, X. 2021. Heterogeneous Major Preferences for Extrinsic Incentives: The Effects of Wage Information on the Gender Gap in STEM Major Choice. Research in Higher Education , 0123456789. Available at: https://doi.org/10.1007/s11162-021-09636-w
  » https://doi.org/https://doi.org/10.1007/s11162-021-09636-w
• EDEN C & ACKERMANN, F. 1998. Making Strategy: The Journey of Strategic Management. Sage Publications.
• FRANCO LA. 2006. Forms of conversation and problem structuring methods: a conceptual development. Journal of the Operational Research Society , 57(7), 813-821. Available at: https://doi.org/10.1057/palgrave.jors.2602169
  » https://doi.org/https://doi.org/10.1057/palgrave.jors.2602169
• FRANCO LA. 2007. Assessing the impact of problem structuring methods in multi-organizational settings: an empirical investigation. Journal of the Operational Research Society , 58(6), 760-768. Available at: https://doi.org/10.1057/palgrave.jors.2602191
  » https://doi.org/https://doi.org/10.1057/palgrave.jors.2602191
• FRANÇOZO R, BELDERRAIN MCN, BERGIANTE N, PACHECO BCS & PIRATELLI CL. 2019. Value-focused thinking na prática: análise do desenvolvimento e aplicações no período (2010-2018). In: LI Simpósio Brasileiro de Pesquisa Operacional. Galoá. Available at: https://proceedings.science/sbpo-2019/papers/value-focused-thinking-na-pratica-analise-do-desenvolvimento-e-aplicacoes-no-periodo-2010-2018-?lang=pt-br
  » https://proceedings.science/sbpo-2019/papers/value-focused-thinking-na-pratica-analise-do-desenvolvimento-e-aplicacoes-no-periodo-2010-2018-?lang=pt-br
• FRANÇOZO R, PAUCAR-CACERES A & BELDERRAIN MCN. 2021. Combining Value-Focused thinking and soft systems methodology: A systemic framework to structure the planning process at a special educational needs school in Brazil. Journal of the Operational Research Society , 1-20. Available at: https://doi.org/10.1080/01605682.2021.1880298
  » https://doi.org/https://doi.org/10.1080/01605682.2021.1880298
• GEORGIOU I. 2008. Making decisions in the absence of clear facts. European Journal of Operational Research, 185(1), 299-321. Available at: https://doi.org/10.1016/j.ejor.2006.12.038
  » https://doi.org/https://doi.org/10.1016/j.ejor.2006.12.038
• GEORGIOU I. 2012. Messing about in transformations: Structured systemic planning for systemic solutions to systemic problems. European Journal of Operational Research . Available at: https://doi.org/10.1016/j.ejor.2012.06.010
  » https://doi.org/https://doi.org/10.1016/j.ejor.2012.06.010
• GHERHES, V, STOIAN CE, FĂRCAŞIU MA & STANICI, M. 2021. E-learning vs. face-to-face learning: Analyzing students’ preferences and behaviors. Sustainability, 13(8), 4381.
• GUIMARÃES RM, ARAÚJO ELEUTÉRIO T & MONTEIRO-DA-SILVA JHC. 2020. Estratificação de risco para predição de disseminação e gravidade da Covid-19 no Brasil. Revista Brasileira de Estudos de População, 37, 1-17. Available at: https://doi.org/10.20947/S0102-3098a0122
  » https://doi.org/https://doi.org/10.20947/S0102-3098a0122
• HU X & HU, S. 2021. Do Colleges Perform the Same Following Developmental Education Reform? The Case of Florida’s Senate Bill 1720. Research in Higher Education , 0123456789. Available at: https://doi.org/10.1007/s11162-021-09641-z
  » https://doi.org/https://doi.org/10.1007/s11162-021-09641-z
• IBGE. 2020. Instituto Brasileiro de Geografia e Estatística. Panorama Das Cidades. Available at: https://cidades.ibge.gov.br/
  » https://cidades.ibge.gov.br/
• IMERAJ L, WILLAERT D, FINNEY N & GADEYNE, S. 2018. Cities’ attraction and retention of graduates: a more-than-economic approach. Regional Studies, 52(8), 1086-1097. Available at: https://doi.org/10.1080/00343404.2017.1362499
  » https://doi.org/https://doi.org/10.1080/00343404.2017.1362499
• JACKSON MC. 1982. The Nature of Soft Systems Thinking: the Work of Churchman, Ackoff and Checkland. In Journal of Applied Systems Analysis, 9, 17-29.
• KEENEY RL. 1992. Value-Focused Thinking a Path to Creative Decisionmaking. Harvard University Press.
• KEENEY RL. 1996. Value-focused thinking: Identifying decision opportunities and creating alternatives. European Journal of Operational Research , 92(3), 537-549. Available at: https://doi.org/10.1016/0377-2217(96)00004-5
  » https://doi.org/https://doi.org/10.1016/0377-2217(96)00004-5
• KEISLER J, TURCOTTE DA, DREW R & JOHNSON MP. 2014. Value-focused thinking for community-based organizations: objectives and acceptance in local development. EURO Journal on Decision Processes, 2(3-4), 221-256. Available at: https://doi.org/10.1007/s40070-014-0032-y
  » https://doi.org/https://doi.org/10.1007/s40070-014-0032-y
• LEE Y, KIRUBARAJAN A, PATRO N, SOON MS, DOUMOURAS AG & HONG, D. 2020. Impact of hospital lockdown secondary to COVID-19 and past pandemics on surgical practice: A living rapid systematic review. The American Journal of Surgery. Available at: https://doi.org/10.1016/j.amjsurg.2020.11.019
  » https://doi.org/https://doi.org/10.1016/j.amjsurg.2020.11.019
• LOGULLO Y, BIGOGNO-COSTA V, DA SILVA ACS & BELDERRAIN MC (2022). A prioritization approach based on VFT and AHP for group decision making: a case study in the military operations. Production, 32. Available at: https://doi.org/10.1590/0103-6513.20210059
  » https://doi.org/https://doi.org/10.1590/0103-6513.20210059
• MAKAGONOV PP, FIGUEROA AR & ESPINOSA SR. 2018. On the Economic Contribution of Specialized Higher Education Institutions to the Development of Monofunctional Cities. Studies on Russian Economic Development, 29(1), 79-85. Available at: https://doi.org/10.1134/S1075700718010094
  » https://doi.org/https://doi.org/10.1134/S1075700718010094
• MANLY CA, WELLS RS & KOMMERS, S. 2020. Who are Rural Students? How Definitions of Rurality Affect Research on College Completion. Research in Higher Education , 61(6), 764-779. Available at: https://doi.org/10.1007/s11162-019-09556-w
  » https://doi.org/https://doi.org/10.1007/s11162-019-09556-w
• MARTINS CB & VIEIRA MM. 2017. Educação Superior e os Desafios no Novo Século: contextos e diálogos Brasil-Portugal. Edu - Unb.
• MINGERS J & ROSENHEAD, J. 2004. Problem structuring methods in action. European Journal of Operational Research , 152(3), 530-554. Available at: https://doi.org/10.1016/S0377-2217(03)00056-0
  » https://doi.org/https://doi.org/10.1016/S0377-2217(03)00056-0
• MOORE, J. 2005. Seven recommendations for creating sustainability education at the university level: A guide for change agents. International Journal of Sustainability in Higher Education. Available at: https://doi.org/10.1108/14676370510623829
  » https://doi.org/https://doi.org/10.1108/14676370510623829
• MORAIS DC, ALENCAR LH, COSTA APCS & KEENEY RL. 2013. Using value-focused thinking in Brazil. Pesquisa Operacional, 33(1), 73-88. Available at: https://doi.org/10.1590/S0101-74382013000100005
  » https://doi.org/https://doi.org/10.1590/S0101-74382013000100005
• ORTEGA F & BEHAGUE DP. 2020. O que a medicina social latino-americana pode contribuir para os debates globais sobre as políticas da Covid-19: lições do Brasil.Physis: Revista de Saúde Coletiva, 30(2). Available at: https://doi.org/10.1590/s0103-73312020300205
  » https://doi.org/https://doi.org/10.1590/s0103-73312020300205
• PARNELL GS, HUGHES DW, BURK RC, DRISCOLL PJ, KUCIK PD, MORALES BL & NUNN LR. 2013. Invited Review-Survey of Value-Focused Thinking: Applications, Research Developments and Areas for Future Research. Journal of Multi-Criteria Decision Analysis, 20(1-2), 49-60. Available at: https://doi.org/10.1002/mcda.1483
  » https://doi.org/https://doi.org/10.1002/mcda.1483
• PATCHING, D. 1990. Practical soft systems analysis. Financial Times/Prentice Hall.
• PAULA K DE A & FARIA TC DE A. 2020. The interiorization of federal Institutions of Higher Education campuses and its impacts on urban space: A case study from the city of Viçosa-MG. Urbe, 12, 1-14. Available at: https://doi.org/10.1590/2175-3369.012.E20190089
  » https://doi.org/https://doi.org/10.1590/2175-3369.012.E20190089
• RITTEL HWJ & WEBBER MM. 1973. Dilemmas in a general theory of planning. Policy Sciences. https://doi.org/10.1007/BF01405730
  » https://doi.org/https://doi.org/10.1007/BF01405730
• ROSENHEAD, J. 1996. What’s the problem? An introduction to problem structuring methods. Interfaces. Available at: https://doi.org/10.1287/inte.26.6.117
  » https://doi.org/https://doi.org/10.1287/inte.26.6.117
• ROSENHEAD, J. 2006. Past, present and future of problem structuring methods. Journal of the Operational Research Society , 57(7), 759-765. Available at: https://doi.org/10.1057/palgrave.jors.2602206
  » https://doi.org/https://doi.org/10.1057/palgrave.jors.2602206
• Rosenhead J & Mingers J(2001). Rational Analysis for a Problematic World Revisited: Problem Structuring Methods for Complexity, Uncertainty and Conflict. Wiley. Available at: https://books.google.com.br/books?id=i30eAQAAIAAJ
  » https://books.google.com.br/books?id=i30eAQAAIAAJ
• TROMBINI MMSL, ROCHA MA DA & LIMA FS. 2020. Avaliação Do Programa Reuni Em Universidades Federais No Brasil. Revista Humanidades e Inovação, 7(6), 91-105.
• VALERO A & VAN REENEN, J. 2019. The economic impact of universities: evidence from around the world. Economics of Education Review, 68, 53-67.
• WHEELER FP & CHECKLAND, P. 2000. Systems thinking, systems practice: Includes a 30-Year Retrospective. The Journal of the Operational Research Society , 51(5), 647. Available at: https://doi.org/10.2307/254200
  » https://doi.org/https://doi.org/10.2307/254200
• ZBOROVSKY GE & AMBAROVA PA. 2018. Higher education as a factor of the cities’ preservation in the rural macro-region. Economy of Region, 14(3), 914-926. Available at: https://doi.org/10.17059/2018-3-16
  » https://doi.org/https://doi.org/10.17059/2018-3-16

Publication Dates

• Publication in this collection
  13 Mar 2023
• Date of issue
  2023

History

• Received
  03 May 2022
• Accepted
  05 Nov 2022