Identification and prioritization of performance for trade decisions indicators: an application of Value Focused Thinking and Analytic Network Process

Identificação e priorização de indicadores de desempenho para decisões comerciais: uma aplicação de Value Focused Thinking e Analytic Network Process

Abstracts

Abstract

The development of a Performance Measurement System (PMS) in an organization is essential in order to adequately monitor operations and enable correct and timely decision-making. However, many organizations implement PMSs and are not successful, mainly because they neglect the design phase, which informs what has to be measured. Designing a PMS is a complex decision problem that involves multiple stakeholders. The objective of this research is to identify and prioritize the commercial performance indicators most valued by the stakeholders in this area in a company in the metal-mechanical segment. The research was divided into two stages, the first being a PO Soft (constructivist) approach: Value Focused Thinking - VFT, to identify performance indicators; and, in the second stage, the application of the PO Hard (Rationalist) Multicriteria Decision Support Method: Analytic Network Process - ANP, for modeling and prioritizing these. As a result, 30 performance indicators were identified, the most important ones being: percentage of target achievement, sales to new customers and percentage of growth for each customer. The results found proved to be capable of assisting the definition of strategy and business management in addition to the identification of indicators not monitored by the organization.

Keywords:
Performance measurement system; VFT; ANP; Commercial decision

Resumo

O desenvolvimento de um Sistema de Medição de Desempenho (SMD) em uma organização é uma questão essencial para acompanhar adequadamente as operações, e possibilitar as tomadas de decisões corretas e a tempo. Porém, muitas organizações implementam os SMDs e não obtém sucesso, principalmente, por negligenciar a fase de projeto, que trata sobre o que medir. Projetar um SMD é um problema de decisão complexo que envolve vários stakeholders. O objetivo desta pesquisa é identificar e priorizar os indicadores de desempenho comerciais mais valorizados pelos stakeholders desta área em uma empresa do segmento metal-mecânico. A pesquisa foi dividida em duas etapas, sendo na primeira aplicada uma abordagem da PO Soft (construtivista): Value Focused Thinking - VFT, para identificação dos indicadores de desempenho; e na segunda etapa a aplicação do Método de Apoio a Decisão Multicritério da PO Hard (racionalista): Analytic Network Process - ANP, para modelagem e priorização destes. Como resultado, foram identificados 30 indicadores de desempenho, sendo os de maior peso: percentual do atingimento da meta, faturamento a clientes novos e percentual de crescimento de cada cliente. Os resultados encontrados mostraram-se capazes de auxiliar a definição da estratégia e gestão comercial além da identificação de indicadores não monitorados pela organização.

Palavras-chave:
Sistema de medição de desempenho; VFT; ANP; Decisão comercial

1 Introduction

Measuring the performance of business processes has become an essential issue in organizations that are constantly challenged to achieve effective and efficient results (Van Looy & Shafagatova, 2016). It plays a crucial role in the rapid provision of information that helps the organization to manage, control, plan and develop the activities carried out (Medel-González et al., 2015).

According to Piratelli & Belderrain (2010), most organizations are not very successful with Performance Measurement Systems (PMSs), especially when it comes to neglecting the design phase. Choong (2018) states, in his review of the literature, that there is no shortage of publications on PMSs, but rather a shortage of research on how the measurement attributes were constructed. Also according to Choong (2014), research on PMSs focuses on more formal aspects in relation to performance measurement, instead of delving deeper into the measurement process: how data is obtained, analyzed, interpreted, communicated and what is the impact of this process in organizational performance itself.

Designing a PMS is a complex decision problem with multiple stakeholders involved. For its resolution, a structuring procedure is necessary, regardless of its level of complexity. Among the problem structuring methods - PSM (Problems Structuring Method) - there is the Value Focused Thinking (VFT), whose approach is focused on value, that is, it seeks to identify the objectives that the stakeholder should use as a basis for decision-making. This approach rests on what the decision maker wants and then figures out how to get it (Pacheco, 2019).

In addition to structuring the problem, the decision maker's preferences allow for a better understanding of its dimensions. The role of the decision maker is not replaced, but based on tools capable of directing the best decision. Hence, Multicriteria Decision Support Methods (MCDM) (Costa & Belderrain, 2009) can be used. The Analytic Network Process (ANP) method deals with dependency within a set of elements (criteria and alternatives) and between different elements. As it is structured in a network, it makes it possible to represent any decision problem without worrying about what comes first or last in the hierarchy, since it does not prioritize only groups or subgroups of elements.

Thus, this article aims to identify the most valued commercial performance indicators for structuring PMSs with stakeholders, using the VFT approach, and prioritizing them using the ANP method. The study was carried out at a company in the metal-mechanical sector that is a leader in the manufacturing of parts (pulleys and couplings). Its commercial area is formed by 8 stakeholders who have the same interests, weights in decisions, but with different areas of activity (states) and different markets (manufacturer of machines and distributors). The organization's difficulty lies in the failure to structure the main performance indicators, causing each stakeholder to waste a lot of time and effort to quantify and work on this information. Another difficulty is that each stakeholder follows principles in identifying the indicators that they think are crucial to that decision or use indicators that have already been built.

In addition to this introduction, the article is divided into the following sections: section 2, with a theoretical framework on PMSs, the VFT approach and its applications in PMS, and the ANP method also focused on PMS applications. In section three, the research methodology is addressed, followed by section 4, with the results. The last section presents the study's conclusions.

2 Performance measurement system

The best-known definition of PMS was proposed by Neely et al. (1995, p. 81): “a set of measurements to quantify the efficiency and effectiveness of actions”. In 1998, Neely complements his definition:

the performance measurement system allows for informed decision-making and action to be taken, because it quantifies the efficiency and effectiveness of past actions through the collection, compilation, ordering, analysis, interpretation and dissemination of appropriate data (Neely, 1998, p. 45).

According to Keathley-Herring (2017), the PMS development process is typically described as having multiple phases: design, implementation and use, with some authors also identifying the PMS review or update process as a distinct phase that provides feedback to improve the system continuously.

Neely et al. (2002) state that organizations generally choose measurements easily obtained with a focus on alternatives, rather than studying appropriate measures for fundamental objectives (thinking focused on value). By not identifying a causal relationship between the indicators, it is not possible to establish a strategic map and, therefore, the measurements are meaningless and blurred.

According to Almeida et al. (2012), several decisions are made daily in organizations with or without formal support methods, and the major concern is with impact. The authors state that problem-structuring methods address this concern and that, regardless of complexity, they must be the first step in the process.

2.1 Value Focused Thinking

For Mingers & Rosenhead (2004) and Mingers (2011), PO soft, through Problem structuring methods (PSM), began to spread with the development of methodologies such as: SSM (Soft Systems Methodology), SODA (Strategic Options Development and Analysis), SCA (Strategic Choice Approach). Authors such as Marttunen et al. (2017), Keisler et al. (2014) and Almeida et al. (2012) encourage the use of the VFT approach as a method for structuring problems.

According to Keeney (1992), the VFT approach seeks to structure the problem based on the strategic values ​​of the decision makers and to identify opportunities in order to improve the set of decision alternatives. The VFT is a way of identifying desirable decision situations and then reap the benefits of those situations. It significantly improves decision making because it is the value that guides the creation of new alternatives and also in the identification of a better decision situation. However, the alternatives are relevant only because they are means to achieve values ​​(Keeney, 1992). The structuring of the problem with the application of the VFT involves: conducting a study on the problem in question; identifying and structuring the decision maker's objectives; identifying alternatives and criteria (Almeida et al., 2014).

It is important to distinguish the types of objectives. They can be classified as strategic objectives, fundamental objectives and means objectives. The strategic objectives correspond to the major objectives of the decision maker; they are those that guide the decision-making of all organizations. The fundamental objectives represent the ends that the decision maker aims at in a decision context, that is, it is these objectives that guide the choice of the decision maker in a given decision context. The means objectives correspond to the way/form to reach a fundamental objective (Keeney, 1992).

It is also necessary to establish how to measure these objectives, by defining appropriate attributes in order to build the value model capable of helping the decision maker to prioritize objectives and measure them (Paiva & Daher, 2016). There are three types of attributes, according to Keeney (1992): natural, constructed and proxy. Natural attributes are the measurements that are commonly used and interpreted by everyone; constructed attributes are those developed to measure the level of achievement of an objective for a specific decision. In many cases, it is necessary to use a constructed measurement, as there are no natural measurements. There are cases in which it is difficult to identify natural attributes or even constructed attributes directly, for these cases, proxy attributes are used (Keeney, 1992).

The use of the VFT approach has been shown to be objective and effective in applications aimed at PMSs. Barclay & Osei-Bryson (2010) applied the VFT approach to identify criteria or performance indicators for evaluating new projects. Their main objective was to assess the real value that these projects provided to organizations. They concluded that there is a strong tendency towards success when the appropriate performance criteria and measures are clear, unambiguous and representative for all those interested in the project.

Kibira et al. (2018) propose a procedure for selecting the key performance indicators (KPIs) for sustainable manufacturing, of what to measure to improve the environmental sustainability of products and manufacturing processes. The VFT approach was applied to identify the necessary metrics. The study contributed to a broader field of the use of KPI for measuring performance in the sustainability of manufacturing systems. The application of VFT was a step towards a standard guide that allows manufacturers to consistently select effective environmental KPIs.

Pacheco et al. (2019b) applied the VFT approach in order to identify the most valued performance criteria, seeking to assist in the management and continuous improvement of a graduate program. According to the authors, the VFT approach goes beyond creating indicators and involves analyzing the program itself to understand its specific needs and requirements. The VFT approach met the established premises, allowing to efficiently identify the objectives as well as the measurable attributes to be used by the program.

2.2 Analytic Network Process

The ANP method is a generalization of AHP, making it possible to analyze the relationships among the criteria and the influences among the alternatives. The ANP does not obey the axiom of independence, and if there is dependence between criteria or influence among alternatives, what is judged is how much one criterion is dependent on another and how much one alternative either is influenced or influences the others (Hernández et al., 2011).

According to Salomon & Montevecchi (1998), in order to deal with the relationships between the elements of the same hierarchical level, the problems have to start to be formulated in networks, not in hierarchies. A network can be defined as a set of clusters, with each nodes that presenting dependency relations between themselves (intra and inter clusters) in any direction (including feedback or feedback).

According to Saaty (2005), the main advantage of ANP over AHP is the possibility of working with problems whose criteria, sub-criteria and/or alternatives have interdependencies, which is very common in practical life. Thus, its result tends to be more reliable (effective) than that of AHP at the expense of efficiency (greater analysis effort as the number of peer-to-peer comparisons between the elements increases).

A difficulty faced in the decision-making process occurs when the problem is not analyzed by an individual, but by a group of people. In addition to the natural complexity of the problem, the interpersonal relationships of the components of the group and the specific objectives of each individual must be foreseen (Costa & Belderrain, 2009). For Piratelli (2010), group decisions tend to be richer than those taken individually because they take into account the plurality of voices of the decision makers of a problem, and consequently promote learning for the group.

According to Saaty & Peniwati (2007), there are two types of approach to group decision: consensual and pluralist. The consensus seeks to avoid major disagreements by means of agreements for the decision (consensus does not mean unanimity, but agreement). The pluralist type seeks to take into account all divergences of views, not avoiding disagreements or conflicts.

In relation to ANP applications in PMS projects, Carlucci (2010) proposed a model, based on the ANP method, to guide managers in the selection of KPIs in the manufacturing process in a sofa company. The model was applied to assess the importance of the existing performance indicators and to prioritize a set of indicators capable of providing adequate information to guide and evaluate management decisions and actions. The ANP method proved to be efficient because it prioritized the criteria and the influence relationship between them.

Piratelli & Belderrain (2010) proposed the construction of a PMS based on the TPP and the ANP method for the modeling and ranking of performance indicators for a Production Engineering course at a Higher Education Institution. 58 performance indicators were identified, distributed in 4 groups. The model proved to be legitimate in accurately reflecting the strengths and weaknesses of the course.

Pacheco et al. (2019a) designed a PMS capable of assisting management and improving the production process of a food production equipment manufacturer and maintenance service provider. The TPP model was applied to align the company's strategy with the production area and the SODA methodology for the construction of the cognitive map and collective reflections of the interested parties. The application of the ANP helped in the modeling and judgments of the PMS. The construction and hierarchy of the PMS allowed the company to develop strategy and management plans for the production process. According to the authors, the multimethodology used proved to be effective when applied.

Moons et al. (2019) proposed the selection of performance indicators to assess the efficiency of logistics processes in operating rooms. The objective was to support hospital logistics managers in making decisions to improve inventory and distribution policies. The selection of KPIs that contributed most to the improvement of the process was established based on the literature and interviews with specialists in hospital logistics. The ANP method was used to calculate the global priority weights, which allowed to select and identify the relevant KPIs to improve the stock and distribution systems. According to the authors, the ANP allowed for the definition of priorities for all performance indicators, considering the perspectives of various stakeholders in a consistent and organized manner. In addition, the approach provided insights to understand the complex relationships in the decision issue and therefore improve the reliability of the corresponding decisions.

3 Methodology

This study was divided into three main phases, as seen in Figure 1. The first phase consists of recognizing the problem in addition to identifying the stakeholders involved. The recognition of the problem comprises the stage of discussion among those involved so that the problem can be dealt with and the concepts are clear to all stakeholders. The identification of the stakeholders involved is a crucial factor. Ackermann & Eden (2011) presented three themes on strategic management of decision makers: identification of stakeholders, the dynamics of relationship between stakeholders and compliance with stakeholder strategies. According to Coelho (2017), the identification of stakeholders is an important step in the decision-making process, since it is necessary to identify people who have the power to define the fundamental and strategic objectives and also make decisions.

Figure 1
Steps of the proposed method. Source: Adapted from Pacheco (2019).

The second phase involves applying the VFT approach. The application of the VFT is divided into four steps: Identification of objectives, hierarchy of objectives, network of objectives and identification of measurable attributes for the objectives. To identify the objectives, individual interviews are carried out using the VFT approach with the stakeholders involved. The facilitator asks questions in order to get stakeholders to think about their values ​​in order to try to identify those that are not so evident.

Regarding the hierarchy of objectives, the Why is That Important? (WITI) is applied, with the statements identified in the previous item (one by one), ranking the objectives by interview. If the answer is that this objective is one of the essential reasons of interest in the situation, it is a fundamental objective. If the answer is that the objective is important to achieve another objective, then it is a means objective (Keeney, 1994).

After the objectives are hierarchized, the facilitator prepares the assembled wish list and structures a network of objectives with the means and fundamental objectives. The network has the purpose of guiding stakeholders in the possible alternatives to be taken to achieve the result.

Once in possession of the network of objectives, the list of measurable attributes is composed, which will be the performance criteria and sub-criteria, extracted and validated by the decision makers from the network. That, in turn, will serve as input for the beginning of the next stage with the ANP.

The third phase consists of the application of the ANP, in which the construction of the model and judgment of the criteria and sub-criteria occur, as well as the presentation of the model results model. For the construction of the model, called the rationalist phase, the model in which the dependency relations between the elements of the clusters will be indicated is constructed. The relations of dependence and feedback between the elements must be in accordance with the network structured by the VFT and legitimized with the participating decision makers. To facilitate modeling, the Super Decisions program will be used.

Regarding the judgments of the criteria and sub-criteria, comparisons are made pair by pair using the Saaty scale. The definition of weights, through the ANP, will determine a strategic direction for the organization's performance. Also in this step, a consistency analysis of the judgments made by the stakeholders will also be carried out.

The results of the model will allow for an ordering of the priorities of the criteria and sub-criteria for the organization's performance, which will allow for better strategic management.

4 Data analysis and results

Due to the high level of competition faced, the organization under study needs guidance on which business indicators to follow in order to: evaluate, compare and correct its performance more efficiently. With a policy far from conventional, the organization has a very lean structure. Each stakeholder is responsible for managing their portfolio and making their decisions. They use the ERP (Enterprise Resource Planning) system to identify indicators that they consider to be the most important. In addition to wasting a lot of time searching for indicators for decision-making, they follow their principles and may not be fully aligned. It is important that the organization understand the success factors and the indicators that affect the results of the business, in addition to everyone involved being aligned with the organization's strategy. It is worth mentioning that there is no conflict of interest of the stakeholders, since each one has a well-defined market and area of ​​activity.

In order to identify the main performance indicators that contribute to decision-making and business management, the VFT approach was applied to seven stakeholders of the organization, all of who work internally in the sales sector. The objective was to apply the VFT approach to all stakeholders in the sector, but due to issues of travel, one of the stakeholders was unable to participate. The interviews took place individually in the period of July 2019, all with the presence of a facilitator, which in this case was carried out by the researcher himself. Such interviews were conducted at the workplace, lasted an average of 45 minutes and were legitimized by the interviewees after being transcribed by the facilitator following the steps of the VFT.

Table 1 shows the guiding questions asked during the interviews. For each answer given to the questions, the following questions were asked: Why is this important? What goals do you want to achieve?

Table 1
Base questions for the interview.

1. What do you think is important to be measured/evaluated in the commercial sector?

2. What performance indicators would give you a competitive advantage at the time of a negotiation?

3. How to identify a product opportunity or a customer?

4. What has been working regarding the performance indicators you currently use? Is there anything that needs improvement?

5. What are the current limitations that hinder you? What needs adjustments?

6. With regard to competitors, what could be improved to give you a competitive advantage?

7. Could you list some desirable and / or undesirable situations (which may be hypothetical)?

8. What do you think needs to be done for the organization to achieve its strategic objectives?

9. Are there any indicators that should be monitored (may be redundant)? And that you don't think monitoring is important?

The strategic objective was identified: Maximization of Profits. The fundamental objectives were defined: to maximize the identification of new customers; maximize the acquisition of new customers; keep active customers (those who patronize ate least once every three months); maximize the sale of new products and minimize commercial expenses.

After the identification of the strategic and fundamental objectives, the facilitator created a network of objectives in order to guide stakeholders in the identification of attributes. A new meeting was held in September 2019 and, with the presence of everyone (in the same room), they were invited to discuss and evaluate the identified objectives (all gathered). Some fundamental and means objectives were modified or rewritten and others were excluded, as they were considered redundant. Then, the final network with the objectives was completed as can be seen in Figure 2.

Figure 2
Network of objectives.

The structuring provided the definition of the strategic objective as being the knowledge of a family of fundamental objectives, which are explained below:

• Maximize profits: It was defined as a strategic objective. It is an efficiency objective, which indicates the gain that the organization manages to generate on the work it develops. There are many factors that can affect the profitability of the organization and these have been identified as fundamental objectives. The fundamental objectives identified were: identifying and winning new customers, selling new products, keeping active customers and reducing commercial expenses.

• Maximize the identification of new customers: there is a need for active sales (outbound), that is, the prospecting itself. Any company carries out active prospecting, but those with qualified performance indicators approach new leads before their competitors and can also prioritize those with the greatest potential for generating business.

• Maximize the acquisition of new customers: It is not enough to just identify new customers; it is necessary to know how to act and make this contact generate sales. For this, it is necessary for stakeholders to identify possible opportunities within these organizations so that they can achieve the sales. The process of gaining new customers is the next step after identifying new customers. There is also receptive prospecting (inbound), in which the customer is the one who contacts the organization.

• Keep active customers: It is not only necessary to find ways to identify and win new customers. It is necessary to keep the current customers and guarantee their satisfaction in order to avoid them switching to the competition. Keeping an active customer for the organization means finding someone who patronizes it at least once in 90 days. After that period, they are classified as inactive.

• Maximize the sale of new products: it is very important to be able to sell more to the same customer and, for that, it is necessary to identify new supply opportunities (increase in the product mix).

• Minimize selling expenses: all selling-related expenses are defined as selling expenses, such as: freight, payment terms, increase in discounts, etc. The organization under study also considers defaults as commercial expenses.

Taking into account the structured network, a list of measurable attributes (Table 2) was also drawn up according to the fundamental objectives, by means of an argument with those involved. It should be noted that any objectives that could be measured were identified. Others of a qualitative nature should be discussed among the stakeholders for alignment within the organization's planning and management process. The identification of the attributes occurred in the same meeting that was held to discuss the network.

Table 2
Identification of measurable attributes.

Fndamental objectives	Attributes	Type of Attribute	Formula to measure the attribute
MAXIMIZING THE IDENTIFICATION OF NEW CUSTOMERS	Registration of new customers every month	Natural	Number of registered potential customers
MAXIMIZING THE AMOUNT OF NEW CUSTOMERS	Visits to customers per month	Natural	Number of non-effective visits
		Natural	Number of effective visits
		Natural	Number of participation or visits to trade fairs
	New customer billing per month	Natural	Billing of new customers (1 purchase)
	Market study by semester	Constructed	City-to- Gross domestic product-GDP turnover ratio
		Constructed	Number of customers/ City GDP ratio
KEEPING ACTIVE CUSTOMERS	Quarterly training	Natural	Number of sales trainings
		Natural	Number of training courses on the organization's product and process
		Natural	Number of market trainings
	Daily customer contact record	Natural	Number of placed calls
		Natural	CRM number (customer relationship management)
	Proposals sent to customers per day	Natural	Percentage of effective proposals (depending on the total number of proposals made)
		Natural	Percentage of canceled proposals (depending on the total number of proposals made)
	Information about each customer per day (active, inactive)	Natural	Days without customer patronage
		Natural	Percentage of growth for each client
	Billing of the customer portfolio per month	Natural	Percentage of goal achievement
	Information about each customer's products per day (last purchase)	Constructed	R$/KG ratio of each customer's product
		Natural	Days without purchase of products
MAXIMIZE THE SALE OF NEW PRODUCTS	Samples sent per month	Constructed	Number of days it took the customer to buy 10x the value of the sample sent
		Natural	Percentage of effective samples (depending on the total number of samples sent)
		Natural	Percentage of failed samples (depending on the total number of samples sent)
MINIMIZE COMMERCIAL EXPENSES	Expenses generated by each customer monthly	Natural	Percentage of order cancellations (relating to billing in a month)
		Natural	Return percentage (depending on billing in a month)
		Natural	Number of complaints
		Natural	Percentage of outstanding financial value (relating to billing in a month)
		Natural	Average discount applied
		Natural	Percentage of freight expenses (depending on invoicing in a month)
		Natural	Average payment term (relating to billing in a month)
	Annual satisfaction surveys	Natural	Customer satisfaction survey score

• GDP: gross domestic product.

In the rationalist phase, indicators were modeled using the Super Decisions software to facilitate the application of ANP. The model that points out the dependency relations between the elements of the clusters was built. The relationships were extracted from the network of objectives and legitimized with the decision makers participating in this phase. The participants in this phase were the 7 stakeholders involved in the constructivist phase. To facilitate the interviews, in the application of the model, the names of the clusters, nodes and subnets that represent them were reduced.

Stakeholders judged the relative importance of clusters and performance criteria through peer-to-peer comparisons. The judgment of the weights was carried out in April/2020, in consensus (with the 7 stakeholders involved in the constructivist phase), by videoconference with the help of the Google Hangouts Meet application.

In all the studies mentioned in this work, none applied ANP by consensus. The author of the research gave preference to this application, because he believed that the appearance of divergent opinions could be debated and help in better decision-making, leading to the redefinition of criteria. Group decisions tend to be richer, more reliable and less subjective than individual decisions because they take into account the plurality of voices.

The Super Decisions software proved to be very useful for the stages of modeling, judgment of stakeholders and, mainly, for the phases of obtaining priority vectors, analysis of matrix consistency and for obtaining results. The software allowed the stakeholders to simulate the behavior of the priority vector of the performance criteria when changing their judgments about the relative importance.

The ANP was carried out in the presence of all stakeholders, who interacted with their preferences and obtained the priority vectors. The trials were conducted by the research author, as a negotiation process to reach collective consensus.

Table 3 presents the prioritized judgments. It is observed that the top five positions in the ranking lead to indicators that are directly linked to financial return. These five indicators add up to 70% of the total weight. They are: percentage of target achievement (FCC1), billing new customers (1 purchase) (FCN1), percentage of growth for each customer (IC2), number of days it took the customer to buy 10x the value of the sample sent (EAE1) and percentage of effective samples (depending on the total number of samples sent) (EAE2). It is natural for stakeholders to prioritize aspects related to immediate financial return, as they are indicators that are closer to achieving the strategic objective.

Table 3
Weight of commercial performance indicators.

Indicator	Description	Weight
FCC	Billing of customer portfolio per month	0.363431
FCN	New customer billing per month	0.169673
IC2	Percentage of growth for each client	0.076854
EAE1	Number of days it took the customer to buy 10x the value of the sample sent	0.069757
EAE2	Percentage of effective samples (depending on the total number of samples sent)	0.035304
VRC2	Number of effective visits	0.035287
IPC1	R$/KG ratio of each customer's product	0.032225
CNC	Registration of new customers every month	0.027255
PCC1	Percentage of effective proposals (depending on the total number of proposals made)	0.024147
IC1	Days without customer purchase	0.019213
EM1	City-to-GDP turnover ratio	0.017438
DGC4	Percentage of outstanding financial value (relating to billing in that month)	0.015162
TR1	Number of sales trainings	0.014028
DGC5	Average discount applied	0.012413
VRC3	Number of participation or visits to trade fairs	0.011115
PS	Annual satisfaction surveys	0.010073
RCC2	CRM (customer relationship management) number	0.008195
DGC6	Percentage of freight expenses (depending on invoicing in that month)	0.008128
EAE3	Percentage of failed samples (depending on the total number of samples sent)	0.007657
TR2	Number of training courses on the organization's product and process	0.005688
DGC7	Average payment term (relating to billing in that month)	0.005582
IPC2	Days without purchase of each product (customer)	0.005371
DGC2	Return percentage (depending on billing in that month)	0.00458
EM2	Number of customers/city GDP ratio	0.004359
PCC2	Percentage of canceled proposals (depending on the total number of proposals made)	0.004025
VRC1	Number of non-effective visits	0.003501
DGC1	Order cancellation amount	0.002933
RCC1	Number of calls made	0.002732
TR3	Number of market training	0.002306
DGC3	Number of complaints	0.00157

The stakeholders involved in the elaboration of this research showed to be very interested, not only with the result of the research, but also with the whole process until the arrival of the proposed solution. At the beginning of the application of the approach, the strategic objective was treated as maximizing commercial performance: planning, control and decision-making. However, by applying the approach, it was realized that all objectives were being directed towards maximizing profitability.

In legitimizing the network of objectives, no conflicts were identified, which reinforces the alignment and engagement of all stakeholders involved. However, a point to note is that the stakeholders with more experience felt more secure during the application of the approach, and thought it was easier to express the objectives. Stakeholders with less experience at the beginning of the application felt insecure.

Regarding the prioritization of the indicators, the indicator “percentage of achievement of the goal” had 36.34% weight. When presenting the results for the validation of the stakeholders, they considered that they had a very high weight in relation to the others. In discussion, it was concluded that the indicator received a high weight because it is an existing indicator and is highly demanded by the organization.

The five most evaluated indicators represent 70% of the total weight and are directly linked to financial return (FCC1, FCN1, IC2, EAE1 and EAE2). In the validation of the results, the stakeholder comment was that the sales/billing ratio is part of the day-to-day activities of the commercial area and therefore has a greater weight. In addition, according to stakeholders, it is these indicators that dictate the market dynamics more quickly.

Another point to highlight is the commercial management indicators, both internal and external (IPC1, CNC1, PCC1, IC1, EM1 and DGC5). These are indicators that lead to greater effectiveness in prospecting, greater effectiveness in negotiations (not increasing discounts in order to close that order by any means necessary) in addition to quick identification when a customer becomes inactive.

With less weight are indicators such as: training, reduction of commercial expenses and customer satisfaction. These are indicators that, according to the stakeholders, are important when achieving the objectives, but which are not yet part of the organization's day-to-day activities. Also according to the stakeholders, the reduction of financial debt (defaulting) is an indicator that tends to have greater weight in future new applications of the ANP, as it has been demanded by the organization (even though it does not yet have a monitoring indicator).

Out of the 30 indicators identified in the study, nine are monitored by most stakeholders (FCC1, FCN1, IC2, IC1, RCC2, IPC2, DGC2, DGC1 and DGC3). Another 3 indicators are monitored by only one stakeholder (IPC1, DGC7, EM2). The other indicators are not monitored by any stakeholder.

The only indicator that the organization monitors and that serves as a plan for the entire organization (such as industrial planning) is the percentage of achievement of the goal. This indicator is planned at the beginning of each year and, with the help of the ERP system, the billing information is extracted.

Regarding the indicators never measured (18 in all), three indicators are among the most important. This demonstrates how much VFT helps in structuring. They are indicators that significantly increase the costs of the organization and do not have a control on their return. A very common practice of the organization is to send samples to the customer to recognize the quality of the product. They often are parts developed especially for the customer that require the development of tooling in addition to special production. The difficulty is to know the percentage of effective samples (which became an order later) and their return (number of pieces that the customer bought). Therefore, it is necessary to find out if the investment was paid for by the number of parts supplied or by the profit margin generated by them.

The number of effective visits is also one of the priority and unmonitored indicators. All the organization's stakeholders plan their own trips, but the effectiveness of these visits has never been measured. The organization considers an effective visit to be one in which contact with the intended person is contacted. An example of an ineffective visit is, for example, one that is not attended by the organization that is intended to be visited, remembering that reporting in CRMs is required whether the visit is effective or not.

The effectiveness of visits indicator was a means created to monitor this task, but other indicators that can’t be measured (qualitative) would be needed to evaluate the real return of this work. For instance, monitoring participation or visits to business fairs. Even though the marketing sector is responsible for bringing a financial return indicator, there are ranges that are not possible to be measured, such as brand visibility.

For the number of trainings, qualitative indicators would also be necessary to demonstrate their real effectiveness. The number of training sessions is an indicator, but the quality of training should be monitored.

Trade-offs were identified in relation to minimizing order cancellation/returns, return values, defaults, complaint numbers and freight costs, concerning the objectives of “gaining new customers”, “keeping customers active” and “maximizing the sale of new products”. As a way to resolve these trade-offs, the attributes associated with the objective “minimize commercial expenses”, were correlated with other objectives. For instance, the attribute order cancellation went from “order cancellation amount” to “order cancellation percentage (based on monthly billing)”. In determining the weights, the criteria of preference of the stakeholders for achieving the objectives are also observed.

5 Conclusions

The PMS project was divided into two stages, the first of which applied the PO Soft (constructivist) methodology: Value Focused Thinking - VFT, which proved to be adequate and contributed to the identification of performance indicators.

In the first step of the first stage, we tried to recognize the problem in addition to identifying the stakeholders involved, gaining a better and more comprehensive view with those involved in the decision-making process. Participation in the research took place with all stakeholders in the commercial area (except for one who was traveling on the day of application). To identify the objectives, individual interviews were conducted using the VFT approach. The same questions were asked to all stakeholders. Stakeholders with more experience, at the beginning of the approach, were more easily able to go a little deeper than stakeholders with less experience.

For the construction of the network of objectives, all the interests, perceptions and preferences established by the stakeholders were considered, generating a consolidated and unique model. The objectives were structured into means objectives and fundamental objectives. The network had the purpose of guiding stakeholders through the possible alternatives to be taken to achieve the final result. The network helped all stakeholders to work in line with the organization's objectives and with performance indicators capable of directing the best decision.

It was found that the structuring of a problem opens a range of options that leads the facilitator, along with the decision makers, to envision new objectives that had not been previously perceived. The fundamental objectives were structured in such a way that they will assist both in the management and in the acquisition of new markets, as well as tend to commercial/financial expenses. The identification of the attributes occurred at the same time as the implementation of the objectives network. In possession of the network of objectives, a list of measurable attributes was created, serving as input for the beginning of the ANP.

The selection results were approved by the stakeholders and their probable benefits have already been noted, namely:

• The applied VFT approach is easy to understand;

• With changes in the organization's strategies, they can easily be applied again;

• The identified performance indicators can be applied immediately, without the need for major adaptations of internal processes;

• Understanding the related indicators will contribute to the achievement of goals and the formulation of sales strategies.

In the second stage, the Analytical Network Process - ANP method of PO Hard (rationalist) was applied. For the application of the method, the Super Decisions software was used, which contributed to facilitating the modeling process, judging according to the Saaty scale and investigating judgment inconsistencies. The application of the ANP method gave PMS greater robustness, as it enabled the performance criteria identified in the constructivist phase to be hierarchized.

An open discussion took place with the same stakeholders involved in the constructivist phase. The subjects were discussed without restrictions, in a creative and active way. They were helpful, committed and sometimes gave up on their preferences, as they were convinced of what was best for the good of the company.

The Super Decisions software proved to be very useful for the stages of modeling, judgment of stakeholders and, mainly, for the phases of obtaining priority vectors, analysis of matrix consistency and obtaining of results. The software allowed the stakeholders to simulate the behavior of the priority vector of the performance criteria when changing their judgments about the relative importance.

The structuring procedure (VFT) and prioritization (ANP) allowed the stakeholders participating in the research to observe the relative importance of each performance criterion within the designed SMD, being possible to identify the management priorities, directing resources or actions that may have greater impact on the organization's performance. In addition, it facilitates stakeholders to make quick and assertive decisions.

The research conducted indicates that, for the specific case of the organization studied and considering the stakeholders involved, the use of the approaches proved to be efficient in the proposed identification and prioritization of performance criteria, as found by Pacheco (2019), in his investigation. The results achieved, by applying the proposed model, will only bring the expected benefits if they are effectively experienced and worked on in a continuous management process.

It is worth mentioning that the results achieved in this research (performance indicators) should not be generalized, as it depends on the stakeholders involved and the strategy of each organization. However, structuring the problem using the VFT approach allows it to be extended to any company that wishes to build a PMS.

It should also be noted that the stakeholders involved should not consider the indicators as a means of analyzing their performance, but rather strategies to improve the performance of the commercial area. Through this research, we expect to collaborate with the organization on which indicators and their priorities for building a PMS.

• Financial support: None.
• How to cite: Dagostin, A. M., Pacheco, B. C. S., Piratelli, C. L., & Costa, V. M. H. M. (2023). Identification and prioritization of performance for trade decisions indicators: an application of Value Focused Thinking and Analytic Network Process. Gestão & Produção, 30, e6122. https://doi.org/10.1590/1806-9649-2023v30e6122

References

• Ackermann, F., & Eden, C. (2011). Strategic management of stakeholders: theory and practice. Long Range Planning, 44(3), 179-196. http://dx.doi.org/10.1016/j.lrp.2010.08.001
  » http://dx.doi.org/10.1016/j.lrp.2010.08.001
• Almeida, A. T., Morais, D. C., Costa, A., & Alencar, L. H. (2012). Decisão em grupo e negociação: métodos e aplicações São Paulo: Atlas.
• Almeida, S. D., Morais, D. C., & Almeida, A. T. (2014). Agregação de pontos de vista de stakeholders utilizando o Value-Focused Thinking associado à mapeamento cognitivo. Production, 24(1), 144-159. http://dx.doi.org/10.1590/S0103-65132013005000037
  » http://dx.doi.org/10.1590/S0103-65132013005000037
• Barclay, C., & Osei-Bryson, K. (2010). Project performance development framework: an approach for developing performance criteria & measures for information systems (IS) projects. International Journal of Production Economics, 124(1), 272-292. http://dx.doi.org/10.1016/j.ijpe.2009.11.025
  » http://dx.doi.org/10.1016/j.ijpe.2009.11.025
• Carlucci, D. (2010). Evaluating and selecting key performance indicators: an ANP-based model. Measuring Business Excellence, 14(2), 66-76. http://dx.doi.org/10.1108/13683041011047876
  » http://dx.doi.org/10.1108/13683041011047876
• Choong, K. K. (2014). Has this large number of performance measurement publications contributed to its better understanding? A systematic review for research and applications. International Journal of Production Research, 52(14), 4174-4197. http://dx.doi.org/10.1080/00207543.2013.866285
  » http://dx.doi.org/10.1080/00207543.2013.866285
• Choong, K. K. (2018). Use of mathematical measurement in improving the accuracy (reliability) & meaningfulness of performance measurement in businesses & organizations. Measurement, 129(1), 184-205. http://dx.doi.org/10.1016/j.measurement.2018.04.008
  » http://dx.doi.org/10.1016/j.measurement.2018.04.008
• Coelho, G. O. (2017). Identificação de políticas de segurança e saúde para a promoção do envelhecimento ativo dos trabalhadores da indústria utilizando value focused thinking (Master’s thesis). Instituto Tecnológico de Aeronáutica, São José dos Campos.
• Costa, T. C., & Belderrain, M. C. N. (2009). Decisão em grupo em métodos multicritério de apoio à decisão. In 15º Encontro de Iniciação Científica e Pós Graduação do Instituto Tecnológico de Aeronáutica (pp. 19-22). São José dos Campos: Instituto Tecnológico de Aeronáutica.
• Hernández, C. T., Marins, F. A. S., & Salomon, V. A. P. (2011). Análise da importância dos indicadores de desempenho da logística reversa mediante a utilização do Analytic Network Process. In Simpósio de Pesquisa Operacional (p. 43). Ubatuba: SOBRAPO.
• Keathley-Herring, H. (2017). An approach to quantify the factors that affect performance measurement system implementation. Engineering Management Journal, 29(2), 63-73. http://dx.doi.org/10.1080/10429247.2017.1298181
  » http://dx.doi.org/10.1080/10429247.2017.1298181
• Keeney, R. L. (1992). Value-focused thinking: a path to creative decision making (p. 432). Cambridge: Harvard University.
• Keeney, R. L. (1994). Using values in operations research. Operations Research, 42(5), 793-813. http://dx.doi.org/10.1287/opre.42.5.793
  » http://dx.doi.org/10.1287/opre.42.5.793
• Keisler, J., Turcotte, D. A., Drew, R., & Johnson, M. P. (2014). Value-focused thinking for community-based organizations: objectives and acceptance in local development. EURO Journal on Decision Processes, 2(3-4), 221-256. http://dx.doi.org/10.1007/s40070-014-0032-y
  » http://dx.doi.org/10.1007/s40070-014-0032-y
• Kibira, D., Brundage, M. P., Feng, S., & Morris, K. C. (2018). Procedure for selecting key performance indicators for sustainable manufacturing. Journal of Manufacturing Science and Engineering, 140(1), 011005. http://dx.doi.org/10.1115/1.4037439
  » http://dx.doi.org/10.1115/1.4037439
• Marttunen, M., Lienert, J., & Belton, V. (2017). Structuring problems for Multi-Criteria Decision Analysis in practice: a literature review of method combinations. European Journal of Operational Research, 263(1), 1-17. http://dx.doi.org/10.1016/j.ejor.2017.04.041
  » http://dx.doi.org/10.1016/j.ejor.2017.04.041
• Medel-González, F., García-Ávila, L., Hernández, C., & Medel-González, M. (2015). Procedimiento para la evaluación del desempeño ambiental: aplicación em el sector energético cubano. Gestão & Produção, 22(3), 463-479. http://dx.doi.org/10.1590/0104-530X101-12
  » http://dx.doi.org/10.1590/0104-530X101-12
• Mingers, J. (2011). Soft OR comes of age but not everywhere! Omega, 39(6), 729-741. http://dx.doi.org/10.1016/j.omega.2011.01.005
  » http://dx.doi.org/10.1016/j.omega.2011.01.005
• Mingers, J., & Rosenhead, J. (2004). Problem structuring methods in action. European Journal of Operational Research, 152(3), 530-554. http://dx.doi.org/10.1016/S0377-2217(03)00056-0
  » http://dx.doi.org/10.1016/S0377-2217(03)00056-0
• Moons, K., Waeyenbergh, G., Pintelon, L., Timmermans, P., & Ridder, D. (2019). Performance indicator selection for operating room supply chains: an application of ANP. Operations Research for Health Care, 23(1), 100229. http://dx.doi.org/10.1016/j.orhc.2019.100229
  » http://dx.doi.org/10.1016/j.orhc.2019.100229
• Neely, A. (1998). Measuring business performance London: Economist Books.
• Neely, A., Bourne, M., Mills, J., Platts, K., & Richards, H. (2002). Strategy and performance: getting the measure of your business Cambridge: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511753695
  » http://dx.doi.org/10.1017/CBO9780511753695
• Neely, A., Gregory, M., & Platts, K. (1995). Performance measurement system design - a literature review and research agenda. International Journal of Operations & Production Management, 15(4), 80-116. http://dx.doi.org/10.1108/01443579510083622
  » http://dx.doi.org/10.1108/01443579510083622
• Pacheco, B. C. S. (2019). Sistemas de medição de desempenho para programas de pós-graduação: uma aplicação em um programa de pós-graduação em biotecnologia (Doctoral dissertation). Universidade de Araraquara, Araraquara.
• Pacheco, B. C. S., Françozo, R. V., Piratelli, C. L., Belderrain, M. C. N., & Amaral, A. C. (2019a). Value Focused Performance indicators for graduate programs. Pesquisa Operacional, 39(3), 497-519. http://dx.doi.org/10.1590/0101-7438.2019.039.03.0497
  » http://dx.doi.org/10.1590/0101-7438.2019.039.03.0497
• Pacheco, B., Piratelli, C., Silva, E., & Belderrain, M. C. (2019b). The design of a performance measurement system for a fruit processing equipment manufacturer. International Journal of the Analytic Hierarchy Process, 11(3), 331-354. http://dx.doi.org/10.13033/ijahp.v11i3.640
  » http://dx.doi.org/10.13033/ijahp.v11i3.640
• Paiva, M. L. U. G., & Daher, S. F. D. (2016). Abordagem VFT para estruturação de problema para melhoria da produção mais limpa em empresa de confecções do agreste pernambucano. In Anais do XLVII Simpósio Brasileiro de Pesquisa Operacional (pp. 437-447). Vitória: SBPO.
• Piratelli, C. L. (2010). Medidas de avaliação de desempenho de uma instituição de ensino superior: uma abordagem de pesquisa operacional (Doctoral dissertation). Instituto Tecnológico de Aeronáutica, São José dos Campos.
• Piratelli, C. L., & Belderrain, M. C. N. (2010). Supporting the design of a performance measurement system with the analytic network process. International Journal of the Analytic Hierarchy Process, 2(1), 44-65. http://dx.doi.org/10.13033/ijahp.v2i1.72
  » http://dx.doi.org/10.13033/ijahp.v2i1.72
• Saaty, T. L. (2005). The Analytic Network Process. In T. L. Saaty & L. G. Vargas (Eds.), Decision making with the analytic network process: economic, political, social and technological applications with benefits, opportunities, costs and risks (pp. 1-26). New York: Springer.
• Saaty, T. L., & Peniwati, K. (2007). Group decision making: drawing out and reconciling differences Pittsburgh: RWS Publications.
• Salomon, V. A. P., & Montevecchi, J. (1998). Método de análise em redes: sucessor do método de análise hierárquica. Produto e Produção, 2(3), 107-117.
• Van Looy, A., & Shafagatova, A. (2016). Business process performance measurement: a structured literature review of indicators, measures and metrics. SpringerPlus, 5(1), 1797. http://dx.doi.org/10.1186/s40064-016-3498-1 PMid:27812439.
  » http://dx.doi.org/10.1186/s40064-016-3498-1

Publication Dates

• Publication in this collection
  17 July 2023
• Date of issue
  2023

History

• Received
  05 Apr 2023
• Accepted
  23 May 2023