A prioritization approach based on VFT and AHP for group decision making: a case study in the military operations

Paper Aims: This paper aims to develop an approach to support group decision making combining methods and tools to a holistic MCDA process. Originality : Authors have been using Value-Focused Thinking (VFT) for structuring problems with different MCDA methods, but there is a lack of a process that defines a clear transition from VFT to those methods. Here we propose a process to fill this gap. Research method : Rich Picture and VFT structure the problem and elicit objectives that become criteria within a decision hierarchy. Analytic Hierarchy Process (AHP) with ratings supports preference elicitation and sensitivity analysis in the judgment weights of decision-makers. Main findings : VFT is effective for eliciting the decision structure to AHP; using weight distribution of stakeholders may affect the results, and the multimethodology approach developed here can deal with group decision making. Implications for theory and practice : The approach developed is effective in complex environments (complex problems and multiple stakeholders) because it focuses on values and defines a process to bring those values into a multicriteria method. Furthermore, sensitivity analysis with the judgment weights of the different stakeholders may be useful in negotiation.

AVAOPs arise from the identification of Operational Deficiencies by Readiness Command subordinate units, and they are formalized through the Identification Form of Operational Deficiencies. To generate an effective flow of resolutions of the identified deficiencies and resource allocation, a prioritization of AVAOPs is necessary. The model to accomplish this objective, used Value-Focused Thinking (VFT) to structure the problem, and Multicriteria Decision Aiding (MCDA). Another research question is: how to combine methods to decisionmaking that reflect stakeholders' values and make the transition from VFT to a method for MCDA? Thus, the development of a model to answer that question emerges as the motivation of this work.
The responsible sector is constituted by a group of specialists and the decision-maker, a general officer, who wants the group's opinion and judgment about the alternatives over the attributes. Therefore, this work explores an approach with methods and tools to surface the subjectivity of the decision and make the evaluation replicable, flexible, and robust. It uses Rich Picture and Value-Focused Thinking (VFT) to structure the problem and define a process to elicit a decision hierarchy from a fundamental objective hierarchy from VFT. The decision hierarchy supports the multicriteria decision making and Analytic Hierarchy Process (AHP) is explored with multiple stakeholders, aiding the decision as Aguarón et al. (2019) propose.
This study objective was to create an approach for supporting group decision making combining methods and tools to a completely MCDA process, as proposed by Belton & Stewart (2002). Thus, facilitating the replicability of a multimethodological approach. It consists of five sections: the first one introduces the objectives and motivation; the second section shows a brief literature review of methods and tools, combined here through a multimethodological process, that supports the approach developed and to it be replicable; section 3 shows the application in the case study; the results are shown in section 4, and section 5 brings a discussion and conclusions about the approach.

Literature review
First, this section presents Problem Structuring Methods (PSMs) and Multicriteria Decision Aiding (MCDA) -the areas in this study -and then, methods and tools that support the intervention.

PSMs
Problem Structuring Methods (PSMs) are a particular area of Soft Operational Research, which considers a situation holistically and ensures sustainability to the proposed recommendations (Ackermann, 2012). They were developed between the 1960s and 1980s to tackle complex problems, solving them at least partially (Mingers & Rosenhead, 2004). To achieve this goal, PSMs shall: 1) enable many alternatives to be brought together into the problem-solution; 2) develop a diagramming that be cognitively accessible to stakeholders with different perspectives and knowledge; 3) operate interactively, allowing adjustments to the representation to the state and stage of discussion among stakeholders and vice-versa; and 4) permit partial or local improvements, rather than global solutions, which marginalize many interests (Mingers & Rosenhead, 2004).
When providing holistic visualization about the problem, methods that structure the situation, make possible the elicitation of potential solutions. They were already extensively reported in the literature (Rosenhead, 1989(Rosenhead, , 2006Mingers & Rosenhead, 2004;Ackermann & Eden, 2011;Ackermann, 2012;Abuabara et al., 2019;Dyson et al., 2021;Abuabara & Paucar-Caceres, 2021) and their analytical assistance is characterized by: multiple actors; different perspectives; conflicting interests; important intangibles; and key uncertainties. Françozo et al. (2021) have presented VFT in a problem structuring context as done here. It will be discussed forward its use to an MCDA model as proposed by Almeida et al. (2015).

MCDA
With many possible solutions and multiple objectives to be achieved, it is necessary to develop a model for decision aiding, which will consider the conclusions from the PSM application. Thus, a method for MCDA shall be used -support the decision-making with multiple objectives, being some of those conflicting (Almeida et al., 2015). Belton & Stewart (2002) present a way of using PSM to structure the problem before specifying alternatives, defining criteria, and eliciting values, because the problem is not "given" for being tackled, it is a messy situation. Figure 1 illustrates two phases of the designed decision support process. Initially, a divergent phase -data collection during the problem structuring -is used to capture the situation complexity and to capture emerging properties of the system. After this moment, a convergent phase takes place and the model is used to put the complexity more objectively, supporting the alternatives assessment and the action plan (Belton & Stewart, 2002;Montibeller et al., 2008).
Four types of problematic are defined by Roy (1996): P.α (choice problematic), refers to the most traditional one, which "the best option" is chosen and is the base for optimization procedures; P.β (sorting problematic) lines up each action to a specific category -the effort is driven to the determination of intrinsic action values, finding your category; P.γ (ranking problematic), help rank actions in order of preferences or build an ordering procedure; e P.δ (description problematic), describe actions and their consequences in a formalized and systematic manner or develop a cognitive procedure. Belton & Stewart (2002) bring two more types: design problematic, to look for, identify, and create alternatives; and portfolio problematic, to choose the group of alternatives for achieving the objectives subject to some restrictions.
This study searches for a prioritization, order the alternatives for preference, thereby, the ranking problematic, considering that the stakeholders might have different weights in the discussion. Next, the tools and methods utilized in both phases, divergent and convergent, will be explored.
2.3. The multimethodological process As Midgley (2015) discusses, methodological pluralism allows "to breach the castle walls" of methodologies, and if they are seen as dynamics, one can learn from the other on an ongoing basis. Similarly, Mingers & Brocklesby (1997) say that one multimethodology principle is to use more than one methodology or part of it, from different paradigms, in the same intervention. Figure 2 shows synthetically the process that will guide this work, inspired by the theory proposed by the authors cited.

Divergent phase
This phase begins with stakeholder identification, which is an important task for all interventions and the decision-making. Ackermann & Eden (2011) developed a tool for stakeholder analysis that may be taken as a robust, leading example of a typical problem structuring approach grounded in interpretivism (Gregory et al., 2020). The analysis is based on the power-interest diagram (Figure 3), which the higher on "y-axis", higher is the level of actor's participation (interest), and the farther right on the "x-axis", higher is the influence (power) that affect the decision, being or not interested in the activities.
There are four quadrants in the Power-interest diagram: "Players" with a high degree of power e high degree of participation (interest) during the process; "Subjects" with high interest, but a low degree of influence over the decision; "Context Setters" with a high degree of power to perform future changes in the organization, however with low interest; and "Crowd" with a low degree of participation and low degree of influence on decision results. The sorting of quadrants is not statistical, it varies depending on the formal and informal relationships in the organization.
To understand the context, and capture the emergent properties, the Rich Picture tool was developed by Checkland (1981) as part of Soft Systems Methodology (SSM). It provides a problematic situation's diagrammatic view, making explicit the system entities, likewise, its structures and stakeholders perspectives over the situation (Georgiou, 2015;Checkland & Poulter, 2020). Rich Picture could be part of an intervention, as Abuabara et al. (2019) did, to start the discussion over the different group perspectives, into a broader view of the properties that emerge from the relationships of the system parts. Similarly, Monk & Howard (1998) said that Rich Picture depicts stakeholders relationships and interests, and, to be effective, it needs five elements: 1) Structure, referring to the context that will be modified, including geographical boundaries, for example, and people involved in the situation (the most important structure); 2) Process, and the situation dynamic, however without depicting the complexity, focusing on what will be transformed; 3) Concerns from stakeholders and different perspectives; 4) The language of the people depicted in it, and 5) Any drawing technique or textual device that suits their purpose.
This construction helps to understand the problem in a systemic manner (Georgiou, 2015), and this is the main reason for its use here. After it, VFT is implemented, shaping the desirable consequences to decision-making.
According to Keeney (1996b), the thinking focused on values is fundamental to everything that people wish to accomplish and shall be the focus of effort when making decisions. This focus on values is the biggest differential of VFT because it concentrates on two main activities: defining what the decision-maker(s) wants and exploring how to achieve this (Keeney, 1996a).  Values are used to assess actual or potential alternatives proposed (Parnell & West, 2008). They can be presented by objectives, which shall be achieved and are composed by a verb on infinitive and an objectthese objectives are "value bits", guiding the decision-making process and creating decision opportunities not visualized before (Keeney, 1996a;. Interactions are needed during the accomplishment of a VFT approach. In those interactions, the analystor facilitator, as defended by Franco & Montibeller (2010) -elicits the objectives using some devices, such as questions that guide the interview, to stimulate the deepest interests and desires. The output is the hierarchy of fundamental objectives and means-ends network. Fundamental objectives are the translation of the desirable consequences that stakeholders are concerned about and are used to assess the alternatives, whereas the means objectives influence, to some degree, the achievement of such fundamental objectives (Keeney, 2008).
An important task traced by Keeney (1996a,;2007; is the fundamental objectives specification, which shall be done until the alternatives could be assessed concerning them (Keeney, 2007). They must be specified until an attribute could be assigned to each of the lower-level objectives. If this isn't possible, the alternatives must be assessed through a natural attribute from a mean objective, however, this is not desirable.
Attributes describe the achievement degree of consequences and are measured in each objective (Keeney, 2007) and consist of three types: natural, construct, or proxy. Natural attributes are a common interpretation: "minimize financial costs" could be the amount of money. On the other hand, construct attributes are developed to measure directly the achievement of an objective. Finally, proxies attribute are used in cases where it is not possible to acquire information about an objective and shall be avoided because they measure the achievement of a mean objective, and this could generate double-counting -one mean objective can be linked in more than one fundamental, for example (Keeney, 1996a).

Convergent phase
Considering the creation of new alternatives, because the AVAOPs flow -new deficiencies are identified during the months -and for any other kinds of problems, it is assumed that the model shall accept the addition of new options, alternatives. And Saaty (2006) in his argument about AHP with ratings, exposes that, if new alternatives arise, the older ranking remains the same, requiring a smaller number of pairwise comparisons, which would be exhausting otherwise (Mu & Pereyra-Rojas, 2018). To assess the alternatives, they must be independent of one another, or, at least, their independence must be assumed, preserving the rank (Saaty, 2008).
AHP implementation can be addressed following four steps (Saaty, 2008): 1) define the problem and explore the kind of knowledge sought; 2) structure the decision hierarchy, starting from the top with decision goal, criteria and subcriteria composed by the other intermediate objectives; 3) construct a set of pairwise comparisons; and 4) use priorities from the pairwise comparison to weigh the priorities in the immediately lower level. The implementation can be summarized as shown in Figure 4 and depicted forward. During the first step, criteria and subcriteria are identified exploring the decision-maker's values and concerns (Parnell & West, 2008). At this moment, the problem is depicted, surfacing the problem complexity and capturing it to manage the situation and understand how decision-makers shall go forward (Belton & Stewart, 2002). As described before, in this step the problem was structured using a Rich Picture, to give a broader and systemic view over the situation. And VFT, to explore the values, and go further into the transition from this step to the next.
In this study, the transition from 1 st step to 2 nd step meant the transformation of the hierarchy of fundamental objectives into the decision hierarchy structure and this process can be visualized in Figure 5. For many authors, those criteria and attributes are synonyms (Keeney, 1996a), but depicting the definitions from this literature review is plausible to say that they are distinct things. Criteria are something that influences the decision, an objective itself, and attribute measures the achievement of an objective.
For example, Almeida et al. (2015) said that in organizations there are many factors related to the decision process, and these factors, when methodologically formulated, are multiple objectives (multicriteria), being only necessary to measure their achievement. Attributes, according to Keeney (2007), measure the achievement of an objective, meaning compliance of the formerly stated. In other words, it can be concluded that criteria are objectives themselves, and a metric linked in it is the attribute -here the ratings of the hierarchy structure are the (constructed) attributes.
In the end, the hierarchy structure defines the global objective, the decision itself, while the criteria and subcriteria are the fundamental objectives from the second level until the lowest, and the ratings reflect the attributes. It is worthy to indicate the main difference from AHP with ratings from traditional AHP: with ratings -intensity levels or categories -, the alternatives are assessed individually concerning each subcriteria or criteria, whereas, when using traditional AHP, it is necessary a pairwise comparison with all alternatives.
Thus, in the 3rd step, when using ratings, there is not a pairwise comparison between alternatives concerning subcriteria or criteria. In this step, these matrices are built with the judgments provided by the decision-makers using Saaty's Fundamental Scale (Saaty, 1977;1990;1994;2001;. Each element is compared concerning the upper level. That is, ratings are compared to subcriteria, the subcriteria are compared to criteria, and criteria are compared to global objective.
Pairwise comparison is illustrated by the matrix A, where A1, A2, and An are being compared: if the element a12 receives the value 3 from Fundamental Scale, meaning that A1 is moderately important if compared to A2, the element a21 receives 1/3.
AHP does not assume transitivity, i.e., if an alternative A is preferred to B, and B is preferred to B, then A is preferred to C. The method admits some inconsistency in the decision-maker judgment process and lower order of magnitude -10 percent -may be considered tolerable (Saaty, 1994).
Transitivity is measured through Consistency Index (CI) (Saaty, 1977;1990;1994;2001;, presented by the equation (λmax -n)/(n -1), where λmax is the largest eigenvalue (mean of elements from product matrix of pairwise comparison matrix and the priority matrix), and n is the number of criteria, subcriteria or ratings and it always has λmax ≥ n. A high index shows that the judgment process is not reliable and there is a high amount of bias and is disturbed, which indicates an unreliable model (Saaty, 1977).
After calculating CI, it shall be compared with the Random Index (RI) -generated from aleatory matrices in 1-9 scale, with forced reciprocity, presented by Saaty (1994), when calculated to square matrices with n elements (Table 1). The Consistency Ratio (CR) is defined as CR=CI/RI. If CR is less than 0.1, it shows a consistent judgment process.
The priorities are defined by a model synthesis, through the sum of weight vectors (Mu & Pereyra-Rojas, 2018).  The 4th step, consisting of judgment aggregation, considered the case of multiple stakeholders using the weighted geometric mean, as explored by Aguarón et al. (2019). Let i k be a pairwise judgment of the i-th stakeholder, whose relative weight in the decision process is i w . The aggregate judgment K is given by Equation 2. Equation 3 gives the boundary conditions for the relative weight distribution. The sensitivity analysis consists of varying the weight scale range of the two stakeholders involved using a percentage scale that varied from 100:0, in which stakeholder #2 judgments are disregarded, to 0:100, in which stakeholder #1 judgments are disregarded. From the literature associated with sensitivity analysis and multicriteria decision analysis, there were no records of such focus on stakeholder weight distribution sensitivity.
Therefore Figure 2 showed each step of the multimethodology proposed here to deal with the MCDA process, and it was depicted in each subsection. Summarizing the model: 1 st ) identify stakeholders; 2 nd ) structure the problem with Rich Picture and VFT; 3 rd ) make the transition from VFT to AHP as presented in Figure 2; 4 th ) apply all steps of AHP; and 5 th ) do the sensitivity analysis.

Developing the model
In this section, the tasks from the multimethodological process, illustrated in Figure 1, are depicted for the application in ranking AVAOPs and model development.

Divergent phase
The first task in this phase is to identify the key stakeholders. Three key stakeholders were chosen for this intervention. They were two majors and one colonel. The colonel is the chief of the sector that decides which AVAOP would be done, thus, the decision-maker. One major is his assessor. He has more experience than his hierarchical superior -the colonel was only two months in his position and the major was two years. This was relevant because it was a motivation to the colonel to invite the major to the decision-making in itself. The other major is the chief of the sector in the Institute of Operational Applications where the AVAOPs are done.
First, they were interviewed individually to construct the rich picture and elicit the objectives. Figure 6 shows the rich picture which was shaped during these interactions: Many emergent properties and relevant systems (Georgiou, 2015) emerged. Nonetheless, the prioritizing was highlighted (near the right top corner), showing the reason why the model developed here is necessary.
The questionnaire used was based on Keeney (1996a). It also made a literature review of strategic legislation, with a bigger focus on DCA 11-45 (Strategic Conception of Air Force 100), because it is the newer strategic legislation and made significant changes in Brazilian Air Force. The workshops and legislation review yield twenty objectives, after taking out redundancy -redundancy is not a deficiency, it is much easier to recognize the desires and concerns when objectives are enumerated, with no restrictions, and although an objective, arising the most hidden interests (Keeney, 2013).
Mean-end network helped to find the fundamental objectives with the WITI Test (Why is it important?). The top-level fundamental objective was "Maximize the Operational Impact". The term "Operational Impact" has a similar meaning to "operational efficiency" shown in legislation. However, the first one was more explored by stakeholders, highlighting their desires and the consequences that could be achieved with the alternatives.
"Operational Impact" appear also on another strategic norm, the Aeronautics Systematic Planning and Institutional Management, as a criterion to start projects in Air Force: "impact straight in Force operationality, through the inter-relationship with Airspace Power Capacity" (Brasil, 2019, p. 30). Considering the attributes generation, the fundamental objective cited before was specified. "Maximize the Operational Impact" was specified (what does this objective mean?) in five more objectives, setting the second level of hierarchy. The process proceeded, stopping in the third level ( Figure 7).
Finally, the ratings were defined finding the attributes linked in fundamental objectives. Thus, the Hierarchy of Fundamental Objectives is transformed into the decision structure, with the process described in section 2, and AHP starts to get the scale's constants for the aggregation value function with the pairwise comparison in criteria, subcriteria, and ratings, concluding with the categorization of alternatives.

Convergent phase
Concluded the problem structuring means that 1 st step of AHP is done, and transforming the fundamental objectives hierarchy to the decision structure consists of 2 nd step -in this work, the second level of fundamental objectives (Figure 7) is the criteria, and the third level is subcriteria -completing the Decision Hierarchy as shown in Figure 8 -and their definitions are synthesized in Tables 2 and 3. The next task is accomplishing pairwise comparisons. Thereby, completing step 3.
Ratings were built upon attributes paying attention to be as careful as Keeney (1996a) argues: "The careful development of a constructed attribute (…) may promote thinking and describe the consequences in a decision situation much better than the "subjective" choice to use a readily available natural attribute" (Keeney, 1996a, p. 104). Figures 9, 10 and 11 present the decision structure with all elements described.
The next task was collecting judgments for pairwise comparisons. Each decision-maker accomplished one hundred judgments. Eight AVAOPs options were assessed to check the model. Half of them were not solved -AVAOP 1, AVAOP 3, AVAOP 4 and AVAOP 8. The AVAOPs content is hidden, although they are real examples of Operational Deficiencies informed to Readiness Command.
The software Excel supported the workshop used to collect stakeholders' judgments and alternative ratings. To remove most of the bias, the AVAOP's options were uncharacterized, thereby, stakeholders did not know which of them were accomplished. Table 4 exemplifies the alternatives rating, already with scale constants after pairwise comparisons.

CRITERIA LABEL DESCRIPTION
Air Force Capacities AFC Future capabilities that will bring operational maturity and, when summed with capabilities from other Forces, will achieve the Defense National Capabilities determined in national strategic legislation (Brasil, 2018

SPP
Born from the concept that is possible to influence the opponent's capability and capacity to fight, without striking directly their forces and troops, developing what is named as dissuasion (Brasil, 2018).

SAE
Capability to control portions from the airspace environment, by a limited period, and prevent the enemy from doing the same (Brasil, 2018).
Command and Control C 2 Capability to afford authority to commanders, searching for the mission accomplishment and control over the results (Brasil, 2018).
Logistic Support LOGS Capability to support the operations, controlling the pace, duration, and intensity of military campaigns (Brasil, 2018).

Information Superiority INFOS
The ability to collect, process, store, scatter, yield, and protect data, denying an adversary this information (Brasil, 2018).
Force Protection FORP Keep troops, resources, facilities, data, and communication security from the ground, preserving the combat power (Brasil, 2018   Going on the additive aggregation value function, considering the evenly-distributed weight of both stakeholders, the ranking in Table 5 shows the results and Figure 12 presents them graphically. The consistency ratio of the aggregated judgments varied from 1.4% to 7.1%, within the 10% suggested by Saaty (1990;1994)      In a first moment, as possible analysis of the results, the facilitator showed that three of four best AVAOPs were not solved yet, an interesting conclusion presenting that a method is needed for this decision, because it has not been taken completely rationally until the moment of this work.
"To evaluate how the results provided by the model vary with the parameters and whether the assumed simplifications affect the results, a sensitivity analysis is essential" (Almeida et al., 2015, p. 224). The sensitivity analysis took place after collecting the stakeholders' judgments. Figure 13 shows how the ranking varies as a function of the weight distribution, as described in section 2.3.2. Figure 13. AVAOP rankings as a function of the weight distribution.
Considering that, in the case study, the three-best ranked AVAOPs would be implemented by the decisionmaker, it is possible to see that the weight system may affect the final list. As implemented, both stakeholders' voice was balanced (50:50), therefore AVAOPs #8, #3, and #6 would be implemented.

Results
The application showed some relevant results about the approach developed: 1) Rich Picture helps with the problem structuring with VFT because the diagrammatic view facilitates the context decision definition. The case studied, helped the stakeholders to see which could be the impact of the decision in the real world, and to surface different decision opportunities.
2) The definition of a clear transition process from VFT fundamental objective hierarchy to AHP with ratings decision hierarchy supported the perception of which values were considered in the decision, helping with the negotiation of alternatives evaluation in the final process of the approach; and 3) Defining weights for decision-makers and showing how the final ranking varies when these weights vary, is a sensitive task. It depends on the group and facilitator's feelings. Thus, it lacks deeper analysis and studies to determine a process that could deal with group heterogeneity and conflicts.

Discussion
Some works are combining SSM and VFT. Françozo et al. (2021), for example, use this combination to structure policies in planning the inclusion of students with special needs in Brazilian Federal High School, however, they do not use an MCDA method to aid the decision. Nevertheless, Abuabara et al. (2019) make the exact combination used here; they did not show how they made the transition from VFT to AHP. Almeida et al. (2015) clearly define in their procedure for resolving problems and building multicriteria models that "for each objective previously established a criterion or attribute has to be proposed" (Almeida et al., 2015, p. 31), although they do not show how people should do it. Here, this gap is filled to make the transition to AHP with ratings, but it is possible to use the same steps with other MCDM/A methods.
The transition process from VFT to AHP to consider the values from stakeholders into the decision hierarchy, and not only for it, but was important to make clear which values were considered. The sensitivity analysis on this ranking problem displays how final results may be affected by the weight distribution of the stakeholders. Firstly, it is important to be as transparent as possible to stakeholders concerning the weight of their judgment. Disclosing clearly how weights are distributed will avoid any dispute that could emerge if the weights were kept secret. Secondly, there are some useful applications of this assessment, such as in private companies.
Shareholders, when having their say in the company's decisions, may use their relative share as means to determine each shareholder's weight. This would allow the application, not only of the AHP but other aggregative methods. A shareholder could evaluate, based on this sensitivity analysis, whether it is attractive to acquire more shares or not to increase their weight in the company's decisions. Aguarón et al. (2019) proposition brought many possibilities for AHP applications, as done here. In this application, the interaction between the DMs has occurred without conflicts because of the hierarchical difference. Nonetheless, Group decision processes are sensitive to the accuracy of the communication process (Almeida et al., 2015). Thus, in replications of the model, this issue must have considerable attention.

Conclusion
The multimethodological process developed here presented a framework to deal with group decision-making holistically. It has involved and engaged stakeholders from the process beginning, with Rich Picture and VFT to structure the problem, eliciting criteria and ratings through fundamental objectives and attributes, exploring workshops to evaluate the alternatives, and with a sensitivity analysis that has brought insights to different groups, building an interactive and adapted method to group decision making, showing a means to deal with in organizations with rigid structure, like military ones and big private companies.
It is important to note that, considering only two stakeholders, there was little concern regarding consistency and coherence -as mentioned by Saaty (1994), the method allows inconsistency to some extent. Future work could explore these areas using, for instance, the precise consensus consistency matrix (PCCM) and solve the optimization problem, as proposed by Aguarón et al. (2019), with more than two stakeholders. Thus, a limitation of this study was not defining how negotiation and conflicts could be managed during alternatives evaluation in this approach, opening a vast field to studies, exploring this phase.
Another interesting take that future works could tackle would be personal bias introduced by stakeholders during negotiations or in the judgments phase. A psychological study could find out whether knowing the weight of their judgment beforehand would have an impact or not on the stakeholder's judgments and final results.