The roles related to performance measurement systems use: a study based on clusters statistics analysis

Os papéis associados ao uso de sistemas de medição de desempenho: um estudo fundamentado em análise estatística de agrupamentos

Abstract:

The performance measurement systems (PMS) have been studied over the last ten years, but this is still a very important research topic for companies and it keep being studied nowadays, because such systems help in decision-making, they allow monitoring and control indicators, and to assist business and operations strategy realization. The recent studies on PMS mostly are on reviewing their concepts and definitions, models, frameworks, design and implementation recommendations, etc. Based on a study oriented to PMS use described by roles to be played, this article aims to discuss the interrelationships between the roles of a performance measurement system, to cluster them and to formulate recommendations for their adoption and proper use. Thus, a survey collected and analyzed the perception of business representatives, who are able to express their agreement levels for PMS roles and use. Data are analyzed by using cluster analysis technique where the roles forms four groups, which can be confronted with the findings of related works from PMS literature. From the groups formed, it is possible to indicate four main domains of recommendations: to assure the alignment between strategic management of operations and its performance, promoting and improving the overall efficiency and effectiveness of the business operations results; to produce positive changes in organizational systems, processes and culture, developing the capacity to manage continuous improvement projects through integrated strategic management systems; to have a customer-oriented design approach; to be orientated and to manage stakeholders’ demands.

Keywords:
Operations strategy; Performance measurement systems; Performance measurement use; Survey; Cluster analysis

Resumo:

Os sistemas de medição de desempenho (SMD) vêm sendo muito estudado ao longo dos anos, porém este é ainda um tema muito importante para as empresas e continua sendo objeto de estudo, pois tais sistemas auxiliam na tomada de decisão, permitem o acompanhamento e controle de indicadores e contribuem para a realização da estratégia de negócios e de operações. Os estudos atuais acerca dos SMD, em sua maioria, versam sobre seus conceitos e definições, modelos, frameworks, recomendações para projeto e implementação, entre outros. Baseado em um conjunto de usos para um SMD descrito na forma de papéis para serem desempenhados, este artigo tem como objetivo principal discutir as inter-relações entre os papéis de um sistema de medição de desempenho, agrupando-os para então indicar recomendações para o seu uso. Para tanto, um survey foi aplicado à representantes de empresas, os quais puderam expressar seus níveis de concordância para os papéis de um SMD. Os dados foram analisados por meio da técnica de análise de cluster, onde os papéis foram agrupados em quatro grupos e comparados com a literatura existente. Dos agrupamentos formados foi possível indicar quatro principais recomendações às empresas, as quais justificam a adoção dos SMD para atingir resultados desejados.

Palavras-chave:
Estratégia de operações; Sistemas de medição de desempenho; Uso de medidas de desempenho; Survey; Análise de agrupamento

1 Introduction

Performance measurement, as a research topic, it has been and continues to be studied by several areas, countless researchers and for a relatively long time, it is still having a significance importance in the research agenda of operations management and accounting. Regardless of the existence of the most sophisticated information technologies that assist in the measurement of performance, with regard to how to measure, the use of performance measurement systems (PMS) is a topic that still requires in depth studies. Measuring the performance of operations, with regard to performance objectives, is a crucial task for organizations, as it assists in decision making, allows the monitoring and control of indicators and guides the operations strategy realization, but how companies are effectively using their PMS?

Nevertheless, the relevance of performance measurement and studies on this topic is justified, because in addition to the diagnosis of the situation, produced by the identification of the use of performance measures, it can also be verified through other uses of performance measurement, alternatives to achieve the organization's performance objectives. In this sense, researchers try to deepen the understanding of how performance measurement happens within organizations in terms of its use, role or functionality (Okoshi et al., 2019; Pinheiro de Lima et al., 2013; Franco-Santos et al., 2007).

It has been recognized that performance management and measurement is fundamental for the effective and efficient management of any business. According to Melnyk et al. (2014) the use of performance measurement and management systems is often recommended to facilitate the implementation of the strategy and improve organizational performance. Performance measurement is treated in this study within the context of strategic operations management. Slack & Lewis (2017, p. 24) define operations strategy as being “[...] the general pattern of decisions that determines the long-term competencies to their contributions to the global strategy, of any type of operation, by reconciling the requirements of the market with the resources of the operations”.

According to Simons et al. (1999), managers rely on performance measurement and control systems to establish direction, make strategic decisions and achieve the desired goals, and in this way the relationship between strategy and performance measurement has been widely discussed and highlighted in literature. Melnyk et al. (2004) recognized the role of PMS systems in operations management, stating that the performance measurement system is ultimately responsible for maintaining the alignment and coordination of operations.

Performance measurement systems, according to Kennerley & Neely (2002) and Hudson et al. (2001), have been gaining great importance for the management of organizations. These systems must be aligned with strategic management in order to maintain and conquer new markets, in addition to improving resource management. Franco-Santos et al. (2012) studied the consequences of contemporary measurement systems, for the authors these systems influence people's behavior, organizational capabilities, performance consequences.

The field in performance measurement has not changed much in the last three decades, but it has been developing in parallel with changes in the global environment, considering, for example, sustainability issues and their environmental and social dimensions. Despite this fact, PMS continue to focus on managing productivity or operations, however, scholars have identified that there are still many pragmatic research gaps that need to be addressed (Mura et al., 2018; Keong Choong, 2014; Bititci et al., 2012)

Franco-Santos et al. (2007) identified that there is no clarity in the definitions of business performance measurement systems. For the authors, PMS definitions have some properties in common: (a) characteristics of PMS; (b) the roles that the PMS plays; and (c) processes that are part of the PMS. More specifically, characteristics are properties or elements that make up the PMS; roles are the purposes or functions that are performed by the PMS; and the processes are the series of actions that combine to constitute the PMS.

Neely et al. (2005) propose an agenda for research on performance measurement. Among the various questions proposed, one of them guides the development of this work as it is based on how each company makes use of its performance measurement system. Thus, and based on studies such as that of Neely et al. (2005) and Franco-Santos et al. (2007) assumes that performance measurement systems, for each company individually, assume different roles.

Pinheiro de Lima et al. (2013) consider that a performance measurement system has eight roles, which this work aims to study. The study by Pinheiro de Lima et al. (2013) developed a theoretical framework based on literature review that organizes and represents the relationships between roles, capacities and performance measurement systems’ recommendations. In this way, the question becomes relevant: How are the roles of a performance measurement system organized and how does its use contribute to organizations?

This context covers this paper, which has as main objective to discuss the interrelationships between the roles of a performance measurement system, grouping them together to then indicate recommendations for their use.

2 Theoretical background

In this section, the main concepts related to the strategic management of operations and the roles of a performance measurement system are presented.

2.1 Strategic management of operations

In the 1970s Skinner (1969, 1974) clearly identified the strategic value of the operations function, highlighting the need for strategic alignment between the business strategy and the operations strategy, particularly the implications of decisions taken in the scope of manufacturing. Skinner spread the concept of manufacturing strategy through a framework that highlights the need to consider the operations function in the development of the strategy. Since then, a series of studies has stressed the importance of the operations function to promote competitive advantage (Prahalad & Hamel, 1990; Wheelwright & Hayes, 1985; Hayes & Upton, 1998; Hayes & Pisano, 1994).

A classic concept that remains until the present day developed by Hayes & Wheelwright (1984) is that the operations strategy is a sequence of decisions, which allow a business unit to achieve a desired competitive advantage over time. Hayes & Wheelwright (1984) developed a maturity model in which roles are defined for the production function. These authors also define aspects of manufacturing considered as world-class related to workforce competence; managerial technical competence; competence for quality management; involvement and participation of the workforce; investment in the development of the production strategy; and the development of flexible operations, capable of responding quickly to market demands and changes.

Leong et al. (1990) propose two complementary models of operations strategy in the process and content dimensions. The former represents how the strategy is developed, implemented and revised, the latter covering the areas of production decision and the competitive priorities defined according to the performance dimensions.

Platts & Gregory (1990) believe that the operations strategy defines how manufacturing will support the achievement of business objectives by providing adequate structural and infrastructural elements to ensure the effectiveness of operations. For Amoako-Gyampah & Boye (2001), the operations strategy consists in the development of competitive differentials, based on the production function, to help achieve long-term competitive objectives.

Slack & Lewis (2017, p.24) define operations strategy as:

[...] the general pattern of decisions that determines the long-term competencies to their contributions to the global strategy, of any type of operation, through the reconciliation of market requirements with the resources of operations. All businesses have markets, all businesses have or allocate resources; therefore, all businesses deal with the reconciliation of markets and resources.

More recently, authors such as Gunasekaran & Ngai (2012) believe that operations management in both manufacturing and service organizations has evolved a lot over the years with changes in market demands. The application of information and outsourcing systems technologies in operations management has significantly changed the landscape of strategic operations management, in addition, environmental and safety awareness also encourages companies to analyze their operations management approach from several perspectives.

Taylor & Taylor (2009) believe that more frequent and current research in the field of operations management is necessary to improve the engagement of the academic world with the practical world in this area. Taylor & Taylor (2009) considered how operations management research can address social and political issues.

2.2 Performance measurement and management systems

Performance measurement is intended to quantify a company's results in terms of efficiency and effectiveness, at the level of business processes and activities performed by people, informing decision-making processes at the right time. It is noteworthy that a company needs a production strategy to guide its manufacturing processes and services, and performance measures allow the development and monitoring of the strategy (Neely et al., 2005; Platts et al., 1998; Lebas, 1995).

A performance measurement system feeds decision-making processes with information on the efficiency and effectiveness of past decisions through the acquisition, compilation, arrangement, analysis, interpretation and dissemination of adequate data. The PMS not only provides data necessary for management to control the various activities of the company, but also influences decisions and organizational behavior (Veiga et al., 2019; Neely, 1998; O’Mara et al., 1998).

According to Folan & Browne (2005) the evolution of studies on performance measurement is divided into four perspectives, namely: recommendations, frameworks, systems and inter-organizational performance measurement. Recommendations are advice related to performance measurement measures or framework. The frameworks assist in the process of building a performance measurement system, can be developed based on the recommendations and are divided into two categories, structural and procedural. Performance measurement systems are the evolution of recommendations and frameworks and the authors indicate three systems as representative in the literature: The Balanced Scorecard (BSC), the Business Process Reengineering (BPR) and the Medori & Steeple (2000) performance measurement system. Finally, and more recently, performance measurement has been studied in an inter-organizational context, where the supply chain and the concept of extended company are generally considered.

The BSC is a framework for measuring and managing performance by Kaplan & Norton (1996a, b) that has four perspectives and four phases in a strategic management process. BPR is a performance measurement system developed based on Bradley’s cube (Bradley, 1996) and focused on the reengineering of business processes. Medori & Steeple (2000) developed a structure that encompasses both the project and the performance measurement audit, this structure works as a system.

Bititci et al. (2012) comment on the changes during the 20th and 21st centuries, ranging from increased industrialization, in which the purpose of performance measurement was the management of productivity, passing to budgetary control and finally, with the emergence of multi-plant organizations more complex and sophisticated markets, PMS was considered for multiple purposes with a focus on strategic control. Artz et al. (2012) confirm that the use of PMS occurs to facilitate decisions related to accounting and accountability, and such associations influence the strategic decision.

The work of Nudurupati et al. (2011) develops a literature review on the performance measurement system and its current state of the art, that is, it contemplates changes in its management models and in its life cycle. The authors analyze the life cycle considering design, implementation, use and assessment of performance measurement. The paper highlights concepts and management about each stage of the life cycle.

Currently, it is considered that a good management control system is associated with good performance and the importance of communicating this performance to stakeholders is emphasized, as it is understood that organizations are obliged to execute and communicate their results as a way of demonstrating contribution to performance. Authors point out that it is extremely important that the processes of customer-oriented performance measurement systems, as a holistic view based on customer needs and project resources is required (Pernot & Roodhooft, 2014; Micheli & Mari, 2014; Wieland et al., 2015; Malavski et al., 2010).

Franco-Santos et al. (2012) studied contemporary performance measurement systems, for the authors, such systems facilitate the development, implementation and assessment of business strategy, focusing on people's decisions, actions on strategic objectives and encouraging an ongoing dialogue on strategic efforts. Contemporary PMS affect communication processes, requiring and providing relevant information that influences the way people think, act and interact. According to Franco-Santos et al. (2012), such PMS influence organizational routines and management practices, changing the way leaders behave. All of these effects have a subsequent impact on performance at all levels.

The performance management system encompasses the process of assessing the differences between the actual and desired results, identifying and signaling those differences that are critical, understanding the cause of them happening, the irregularities that have occurred, and, when necessary, the system must perform monitoring of actions aimed at filling significant performance gaps. The system must be capable of being operated as a thermostat, for measurement purposes, and also to allow high-level functions, such as questioning the organization's standards, assumptions and strategies to be discussed (Melnyk et al., 2014; Argyris, 1977).

Koufteros et al. (2014) describes that a performance measurement system can also be characterized as a management control system that incorporates a structured framework specifying the main financial and non-financial performance metrics. From a theoretical point of view, a PMS can be described as an ambidextrous system, as it incorporates both mechanistic and organic elements.

Authors such as Nudurupati et al. (2016) talk about the changes that the world has been undergoing, especially in the digital issue, and the impacts on performance measurement. The authors point out two important issues that characterize contemporary PMS, one being the constant change in the external environment that forces PMS to be more dynamic and the second the different varieties and volumes of data that organizations have to deal with to create competitive advantage. The authors also identify four emerging themes that impact performance measurement and management, namely, collaboration and co-creation, sustainable agenda, big data and internet of things and organizational culture.

2.3 Performance dimensions

According to Slack & Lewis (2017), an operation requires a set of strictly defined objectives, these are the performance objectives or performance dimensions that apply to any type of operation.

Slack & Brandon-Jones (2019) consider the following performance dimensions for manufacturing:

• Quality in offering products in accordance with the appropriate project specifications. Quality suitable for the purpose includes two levels, one of which is service or product specification, the other is whether the operation achieves compliance with this specification;

• Flexibility to adapt the operation whenever necessary and with the required speed. It is a concept that in operations can be understood as the ability to assume different states, that is, to adopt different positions or to do things in different ways;

• Speed ​​in making the time interval between the beginning of the production process and the delivery of the product to the customer as short as possible. It can be related to events, such as the moment when the customer placed the order until delivery or it can be used internally in the operation, for example, the time for a material to enter a process and leave ready;

• Reliability in fulfilling the promises of delivery time or honor in the delivery time given to the customer. It is the other half that approaches delivery speed, these two objectives are generally related;

• Cost to offer products with lower prices than those of the competition. It is, for companies that compete for price, the most important performance objective. For other companies, it is also a performance objective considered crucial, as any company seeks to reduce costs;

• Innovation as new products are designed and launched, developing these faster than competitors.

Recently, sustainability is being used to frame performance objectives in terms of environmental, social and economic perspectives. According to Pasqualini Blass et al. (2017) and Epstein et al. (2015), there is a need to approach operations performance in a sustainable integrated perspective.

2.4 PMS roles and functions

Many authors have studied the roles of performance measurement systems, this theme was, for example, a research agenda proposed by Neely et al. (2005). Since then, authors such as Pinheiro de Lima et al. (2013) and Franco-Santos et al. (2007) have studied the roles of PMS.

In order to assist researchers in the process of identifying and selecting the roles of performance measurement systems Franco-Santos et al. (2007) proposed five different categories of PMS functions. These are: (1) Measuring performance: this category encompasses the role of monitoring progress and measuring/evaluating performance; (2) Management strategy: this category comprises the functions of planning, formulation of strategies, implementation / execution strategy and focus of attention / alignment; (3) Communication: comprises the functions of internal and external communication, benchmarking and compliance with regulations; (4) Influence on behavior: this category includes the functions influence on behavior through gratification or compensation, management and control of relationships; (5) Learning and improvement: comprises the functions of feedback, double circuit learning and performance improvement.

Pinheiro de Lima et al. (2008, 2013) consider that a performance measurement system has eight roles, which are defined by:

• P1: Implement a functionality related to strategic management in the operations strategy management system, promoting the joint improvement of operations’ efficiency and effectiveness indicators in achieving business objectives;

• P2: Be responsible for deploying the strategy and monitoring the business results;

• P3: Produce a positive change in organizational systems and processes;

• P4: Develop training to manage continuous improvement processes, through the implementation and management of an integrated strategic operations management system;

• P5: Produce a positive change in the organizational culture;

• P6: Produce a closer understanding of the needs of the market, and thus create value that is perceived by customers;

• P7: Show how the design specifications lead to the desired results;

• P8: Meet external demands not directly managed by the organization.

According to a Delphi study carried out by Pinheiro de Lima et al. (2013, p.531), these roles are divided as follows: Roles 1 and 2 “[...] implement the strategic management functionality in the strategic operations management system, providing the system with the improvement joint efficiency and overall business operational effectiveness [...]“, roles 3 through 6” produce a positive change in organizational systems and processes “and roles 7 and 8” show how project requirements and external system contingencies lead to desirable results ”.

The grouping proposed by Pinheiro de Lima et al. (2013) is consistent with the proposal by Franco-Santos et al. (2007), who says that role 2 refers to “measuring performance”, roles 1 and 2 with “strategic management”, roles 2 and 5 with “communication”, roles 3, 5, 7 and 8 “influence behavior” and roles 1 and 4 with “learning and improvement”. Although organizational learning is not explicit in the PMS roles statement, it could be treated as a basic resource that supports operations of developing the strategic management system.

2.5 Conceptual framework

All the concepts used are compiled in a conceptual framework, which comprises all the theoretical basis used in this research and positions the focus of this study. This framework is presented in Figure 1.

Figure 1
Conceptual framework.

The framework consists of several areas, one of which is the macro area understood as a business environment. Within the business environment, some processes take place, such as the transformation of the business strategy into company results, for which the business strategy is deployed in the operations strategy that defines some performance objectives (quality, flexibility, reliability, cost, innovation and speed of delivery) to be dealt with in the planning of operations, after this treatment emerges to the real performance dimensions. The planning of operations is again linked to the strategy via a performance measurement system (confronting the importance of the dimensions of performance versus actual performance). The area of ​​performance measurement systems is that which supports the strategy, seeking assistance in reviewing the general strategy and supporting the operations strategy. Each of the eight roles of a performance measurement system defined by Pinheiro de Lima et al. (2008, 2013) is positioned in the conceptual framework according to its functionality and indicated by P1 through P8.

3 Research design

In this section, the methodological approach, research strategy, techniques used for collecting and analyzing data are presented.

According to Creswell (1994), Amaratunga & Baldry (2002) and Creswell & Clark (2006) the quantitative approach is the most appropriate to test a theory, when it is an exploratory study and, in this context, it is based on this approach to analyze and test the impact of a set of performance indicator roles on industrial systems.

The most striking feature of the quantitative approach is the fact that it measures the research variables, and this is often the justification for choosing this approach. The research strategy adopted is the survey. This type of methodological approach, in general, evaluates a significant sample of a problem to be investigated in order to draw conclusions (Bryman & Bell, 2015; Martins, 2012).

According to Hair et al. (2009), in multivariate data analysis, the technique used was cluster analysis, also called cluster analysis. For the identification of homogeneous observation groups, this is a widely used technique. Cluster analysis aims to develop significant subgroups of individuals or objects, defined based on the similarity between entities, that is, this technique is used to identify groups. Within the multivariate analysis of data, this technique classifies objects according to certain characteristics, in such a way that these objects are similar to each other within a group.

Pohlman (2007, p.325) considers that “[...] the technique serves to divide a set of objects into two or more groups, based on the similarity between them, for a given set of characteristics”. Still according to Pohlman (2007), cluster analysis can be seen as an approach to building a six-steps model, these steps are not independent and, consequently, it is necessary to return to previous steps to correct and improve the others (see Table 1).

Table 1
Cluster analysis steps.

Steps	Description
Step 1: Cluster analysis objetives	By forming homogeneous groups, it is possible to achieve any of the following objectives: (1) Taxonomic description; (2) Data simplification; (3) Data simplification.
Step 2: Research design	Three questions must be addressed: (1) are there outliers and, if so, should they be excluded? (2) how should the measure of similarity between objects be? (3) Should data be standardized?
Step 3: Cluster analysis assumptions	The focus should be kept on two critical points: sample representativeness and multicollinearity.
Step 4: Cluster formation	One must choose the algorithm (hierarchical or non-hierarchical) used to form the groups and then make a decision on the number of groups to be formed.
Step 5: Cluster analysis	It involves examining each group, in view of the set of variables chosen, to name or assign an identification that adequately describes the nature of the groups.
Step 6: Cluster test and validation	In the evaluation, it must be ensured that the solution is representative of the general population and, consequently, generalizable to other objects and stable over time.

• Source: Pohlman (2007).

The research data collection was structured in three stages, in the years 2008 (with 104 respondents), 2011 (with 100 respondents) and 2013 (with 101 respondents), the applications of the years 2011 and 2013 had improvements identified in previous applications and the combination of the three data collections resulted in a sample of 305 respondents. The level of significance of the ANOVA test applied to the set of the three collections is greater than 0.05. In this study, the majority (83%) of the significance indices (Sig.) have a value greater than 0.05, making it possible to accept the null hypothesis that the averages are equal and the three samples from 2008, 2011 and 2013, can be considered as one.

In order to carry out the three surveys, a physical questionnaire (interviews) was applied to representatives of companies in Parana state/Brazil who participated in a continuing education program of a higher education institution in the state, these representatives were from the courses in Production Engineering, Quality, Logistics, Lean Six Sigma among other courses of the Graduate Program of the Polytechnic School and Lean Academy of a higher education institution. The applied questionnaire has two parts:

• Part 1: Identification. Practice area, number of employees, annual sales, plant size, legal status and origin of the headquarters;

• Part 2: PMS roles survey. Considering the dimensions of performance, for each of the eight roles, apply a degree of agreement according to a 5-point Likert scale.

The present study works with the first and second blocks of the questionnaire. In the second block, which is the main focus of this study, the approach was on the roles of performance measurement systems developed by Pinheiro de Lima et al. (2008, 2009). Respondents were able to express a degree of agreement on these roles in each performance dimension according to a 5-point Likert scale, being (1) Strongly disagree; (2) Disagree; (3) It is indifferent; (4) Agree or (5) Strongly agree.

Respondents were able to express the degree of agreement for each of the roles as shown in Table 2:

Table 2
PMS roles survey.

To implement a functionality related to strategic management in the operations strategy management system, providing the system for joint improvement of operational efficiency and effectiveness indicators in achieving business objectives.	[1]	[2]	[3]	[4]	[5]
To be responsible for deploying the strategy and monitoring the business results.	[1]	[2]	[3]	[4]	[5]
To produce positive change in organizational systems and processes.	[1]	[2]	[3]	[4]	[5]
To develop training to manage continuous improvement processes, through the implementation and management of an integrated strategic operations management system.	[1]	[2]	[3]	[4]	[5]
To produce a positive change in organizational culture.	[1]	[2]	[3]	[4]	[5]
To produce a closer understanding of market needs, and thus create value that is perceived by customers.	[1]	[2]	[3]	[4]	[5]
To show how design specifications lead to desired results.	[1]	[2]	[3]	[4]	[5]
To meet external demands not directly managed by the organization.	[1]	[2]	[3]	[4]	[5]

• Source: Pinheiro de Lima et al. (2008, 2013).

Once the methodological concepts and questions are defined, some results and discussions are presented in relation to parts 1 and 2 of the survey questionnaire.

4 Results

The results that describe the survey companies’ profile is related to the first part of the questionnaire, Figure 2 shows the sector of activity and number of employees in percentage. Most of the sectors of activity present in the survey are automotive (17%), metal-mechanics (12%), food (11%), electro-electronics (7%), furniture (7%), service (5%), civil construction (5%). Regarding the number of employees, 54% of the companies have more than 500, 21% have 50 to 200, 17% have 200 to 500 and 8% have less than 50 employees.

Figure 2
Surveyed companies.

Figure 3 presents the respondents' profile in terms of annual sales, plant size, legal status and company origin in terms of percentage. With regard to annual sales, 69% of companies earn more than five million Reals, 17% between two million and five million, 9% from five hundred thousand to two million and only 5% of companies earn less than five million. With regard to the size of the plant 62% of the companies have plants over 5,000 m², in 23% the plants have between 1,000 m² and 5,000 m², 11% between 200 m² and 1,000 m² and finally only 4% of the companies have less of 200 m². For the legal situation 52% are independent companies, 25% subsidiaries of large companies and 23% service or manufacturing unit of a large company. Finally, 67% have their country or continent of origin as Brazil, 21% Europe, 7% North America, 3% South America, 1% Asia and 1% are among others.

Figure 3
Companies revenue, size, origin and type.

Data from the second block of the questionnaire were analyzed using cluster analysis with the objective of taxonomic description of the roles of the PMS, as it sought to compare the groups formed with others already existing in the literature. As the data are considered interval, the measure used was the Square Euclidean Distance and according to Pohlman (2007) this type of measurement is recommended for the Centroid and Ward methods.

For this study, Cronbach's Alpha was also calculated, which resulted in the value 0.896082934, which proves the validity and understanding of the questionnaire, since values ​​above 0.7 for Cronbach's Alpha already prove these statements according to Hair et al. (2009).

It is important to highlight that the respondents to the questionnaire in this survey assigned a degree of agreement for each role of a PMS with respect to each dimension of performance, the Centroid and Ward methods were applied to a general sample with the sum of all dimensions of performance and was also performed for each dimension separately (see Figure 4).

Figure 4
Performance dimensions surveyed.

As the results obtained for each dimension separately, for the most part, are the same obtained for the general (which is the sum of all dimensions), in this study the results of the general will be presented, which is the one that best represents the entire sample. Two different groupings are proposed for this study, first the choice of the number of clusters to be formed for the general and for each dimension is 3, based on a Delplhi study carried out by Pinheiro de Lima et al. (2013) who identified three groupings for this same set of roles. Table 3 shows the three groups formed with the Centroid and Ward methods.

Table 3
Grouping (3).

General		General
Centroid		Ward
Case	3 clusters	Case	3 clusters
1:P1	1	1:P1	1
2:P2	1	2:P2	1
3:P3	1	3:P3	1
4:P4	1	4:P4	1
5:P5	1	5:P5	1
6:P6	1	6:P6	2
7:P7	2	7:P7	2
8:P8	3	8:P8	3

Considering the three groups, for the centroid method, the roles of 1 to 6 comprised the first group, role 7 the second and role 8 the third. For the Ward method, roles 1 through 5 comprised the first group, roles 6 and 7 the second group and role 8 the third.

From these clusters, a large agglomeration between roles 1 to 6 was identified and, thus, the analysis was redone considering 4 clusters to be formed. Table 4 presents the four groupings formed with the Centroid and Ward methods for the general/overall performance dimensions.

Table 4
Grouping (4).

General		General
Centroid		Ward
Case	4 clusters	Case	4 clusters
1:P1	1	1:P1	1
2:P2	2	2:P2	1
3:P3	1	3:P3	2
4:P4	1	4:P4	2
5:P5	1	5:P5	2
6:P6	1	6:P6	3
7:P7	3	7:P7	3
8:P8	4	8:P8	4

Considering the four groups, for the centroid method the roles of 1, 3, 4, 5 and 6 comprised the first group, role 2 the second, role 7 the third and role 8 the fourth. For the Ward method, the roles of 1 and 2 comprised the first group, roles 3, 4 and 5 the second group, roles 6 and 7 the third and role 8 the fourth.

Considering studies such as that of Pinheiro de Lima et al. (2013) and Franco-Santos et al. (2007), the group that shows the most coherence is the one with a pre-determined number of groups (4) and the Ward method, which the dendrogram is shown in Figure 5.

Figure 5
Dendrogram.

Presented the results of the survey, they could be analyzed and discussed regarding theory and practice of PMS use.

5 Discussion

PMS roles clustering shown to be totally coherent with the grouping formed in the Dehphi study by Pinheiro de Lima et al. (2013). In the Delphi study, where the pre-determined number of groups was 3, there was a greater agglomeration between roles 3 and 6, which was resolved by pre-determining 4 groups for the survey. Table 5 presents this comparison and summarizes the interrelationships of the roles, where P1 and P2 are more related to strategy, P3, P4 and P5 connected to producing positive changes in management systems, processes and organizational culture, P6 and P7 are more focused on customer approaches and P8 is related to stakeholder demands. The first column of the table refers to the eight roles of a PMS, the second presents an interpretation of the groupings identified by Pinheiro de Lima et al. (2013) made by the authors themselves and the third column presents an interpretation of the groupings identified in this study.

Table 5
Comparing the Performance Measurement Systems (PMS) grouping results.

PMS roles	Delphi Study	Survey - General
P1	To implement the strategic management functionality in the strategic operations management system, providing the system with the joint improvement of overall business operational efficiency and effectiveness.	To be in alignment with strategic management of operations promoting and improving the overall business operations’ efficiency and effectiveness.
P2
P3	To produce positive change in organizational systems and processes.	To produce positive changes in organizational systems, processes and culture, developing the capacity to manage continuous improvement projects through the integrated strategic management system.
P4
P5
P6		To have a customer-oriented design approach (Total Quality Management - TQM, Lean enterprise, 6sigma, Design for Manufacturing - DFM, Quality Function Deployment - QFD).
P7	To show how design requirements and external system contingencies lead to desirable results.
P8		To be orientated to stakeholder demands.

Making a comparison with the existing literature, cluster 1 can be compared to the study by Franco-Santos et al. (2007) when the authors classify PMS with the functionality of strategic management, performance measurement and communication. The second group is linked to the functionalities that influence behavior, learning and improvement and communication. The third group and the fourth group are linked to the influence on behavior.

Neely et al. (2005) say that a PMS is intended to quantify the efficiency and effectiveness of an organization. Neely's classic concept is fully consistent with the first grouping in the sense that strategic alignment leads to operational efficiency and effectiveness. Bititci et al. (2012) in his study on the evolution of PMS shows that one of the main themes of study is on the role of strategic control of PMS. Pernot & Roodhooft (2014) highlight in their article that a good management control system is associated with good performance.

A measurement system must have the functionality to produce a positive change in the organizational culture, systems and processes, according to the second group. This statement is fully consistent with the statement by Amaratunga & Baldry (2002), because for the authors, a strategic PMS must exert exactly such functions.

The third grouping is consistent with the statements of Bititci et al. (2012) who say that since the sixties, where they started studies on performance measurement, the PMS has the functionality to promote customer satisfaction, meeting, for example, the dimensions of flexibility, quality and time. Wieland et al. (2015) highlight the importance of the processes of customer-oriented performance measurement systems, for the authors a holistic view is necessary based on the needs of the clients and project resources.

Regarding the last group that contains only P8, authors like Micheli & Mari (2014) say that, in fact, organizations are obliged to execute and communicate their results to stakeholders as a way to demonstrate their contribution to performance. Pernot & Roodhooft (2014) state that companies can adapt their control systems to adjust the relationship with suppliers, for example, seeking to improve performance, in addition, the study demonstrates the importance of control over supplier management, as formal controls able to overcome operational difficulties.

Artz et al. (2012) state that the use of PMS occurs to facilitate the decision for accounting / accountability, such associations influence the strategic decision. This statement is consistent with the first and last groupings identified in this study.

To conclude this discussion, Chenhall & Langfield-Smith (2007) show a multidisciplinary view on performance measures that go beyond financial measures, focused on stakeholders, to a view that considers operations management, marketing and human resources and strategy. The marketing vision involves criteria related to the customer, such as satisfaction, added value recognized by the customer. Human resources involve a strong focus on the development of performance measures to assess organizational performance aimed at employees, these measures are related to both remuneration, training issues and adaptation to organizational culture, for example. The strategy-oriented view is the key issue of performance measurement systems, which is intended to assist in practices that help in planning the long-term direction of organizations and implementing these plans. The multidisciplinary view identified by Chenhall & Langfield-Smith (2007) is also fully consistent with the groupings identified in this study.

This study sought to discuss the interrelationships between the roles of a performance measurement system, grouping them together to then indicate recommendations for their use. The interrelationships were identified via cluster analysis and confirmed through the existing literature. The recommendation for the use of PMS is clear when identifying that it assists in strategic management and leads to desired results.

6 Conclusion

Seeking to answer the question ‘How are the roles of a performance measurement system organized and how does its use contribute to organizations?’ and from the results obtained, it is possible to verify the contribution of this study for both company managers and academia. For the academy, the study contributes by verifying that the four groupings of the PMS roles are consistent with the literature and other studies previously carried out, such as that of Pinheiro de Lima et al. (2013), thus allowing the conclusion that this is a good representation of the uses of systems and performance measurement.

Regarding the contribution of this study to organizations, it is clear that when a performance measurement system is implemented, it has four main purposes: implementation, preservation and review of the strategy; driving, directing and reaching determined goals and promoting benefits for the organizational system; meeting customer demands; and meeting the demands of stakeholders.

The identified clusters contribute to operations managers by briefly clarifying the roles related to the use of a PMS and how it is possible to achieve the full efficiency and effectiveness of organizations, especially with regard to their performance objectives. In addition, this study allows us to say that the use of PMS is of essential importance for companies, as it provides that strategic control of operations leads to desired results.

As a limitation of this study, it is important to emphasize that the analysis of the results may be subject to the competence and experience of the survey respondents, who may not be directly involved with the strategy or operations.

As a proposal for future work, it is suggested a temporal analysis of the survey results and the realization of filters in relation to the companies' profile, in order to identify if there are differences in the organization of the SMP roles for each profile. It is also suggested to apply other multivariate analysis techniques for possible different interpretations of the results.

• Financial support: CNPq: 308137/2018-3 e 409350/2018-4
• How to cite: Rocha, L. M., Pinheiro de Lima, E., Gouvêa da Costa, S. E., Sant’Anna, A. M. O., & Angelis, J. J. (2020). The roles related to performance measurement systems use: a study based on clusters statistics analysis. Gestão & Produção, 27(4), e2708. https://doi.org/10.1590/0104-530X2708-20

References

• Amaratunga, D., & Baldry, D. (2002). Moving from performance measurement to performance management. Facilities, 20(5-6), 217-223. http://dx.doi.org/10.1108/02632770210426701
  » http://dx.doi.org/10.1108/02632770210426701
• Amoako-Gyampah, K., & Boye, S. S. (2001). Operations strategy in an emerging economy: the case of the Ghanaian manufacturing industry. Journal of Operations Management, 19(1), 59-79. http://dx.doi.org/10.1016/S0272-6963(00)00046-2
  » http://dx.doi.org/10.1016/S0272-6963(00)00046-2
• Argyris, C. (1977). Organizational learning and management information systems. Accounting, Organizations and Society, 2(2), 113-123. http://dx.doi.org/10.1016/0361-3682(77)90028-9
  » http://dx.doi.org/10.1016/0361-3682(77)90028-9
• Artz, M., Homburg, C., & Rajab, T. (2012). Performance-measurement system design and functional strategic decision influence: the role of performance-measure properties. Accounting, Organizations and Society, 37(7), 445-460. http://dx.doi.org/10.1016/j.aos.2012.07.001
  » http://dx.doi.org/10.1016/j.aos.2012.07.001
• Bititci, U., Garengo, P., Dorfler, V., & Nudurupati, S. (2012). Performance measurement: challenges for tomorrow. International Journal of Management Reviews, 14(3), 305-327. http://dx.doi.org/10.1111/j.1468-2370.2011.00318.x
  » http://dx.doi.org/10.1111/j.1468-2370.2011.00318.x
• Bradley, P. (1996). A performance measurement approach to the reengineering of manufacturing enterprises (Doctoral thesis). CIMRU, NUI Galway, Ireland.
• Bryman, A., & Bell, E. (2015). Business research methods (4th ed.). Oxford: Oxford University Press.
• Chenhall, R. H., & Langfield-Smith, K. (2007). Multiple perspectives of performance measures. European Management Journal, 25(4), 266-282. http://dx.doi.org/10.1016/j.emj.2007.06.001
  » http://dx.doi.org/10.1016/j.emj.2007.06.001
• Creswell, J. W. (1994). Research design – qualitative and quantitative approaches London: Sage.
• Creswell, J. W., & Clark, V. L. P. (2006). Designing and conducting mixed method research London: Sage.
• Epstein, M. J., Buhovac, A. R., & Yuthas, K. (2015). Managing social, environmental and financial performance simultaneously. Long Range Planning, 48(1), 35-45. http://dx.doi.org/10.1016/j.lrp.2012.11.001
  » http://dx.doi.org/10.1016/j.lrp.2012.11.001
• Folan, P., & Browne, J. (2005). A review of performance measurement: towards performance management. Computers in Industry, 56(7), 663-680. http://dx.doi.org/10.1016/j.compind.2005.03.001
  » http://dx.doi.org/10.1016/j.compind.2005.03.001
• Franco-Santos, M., Kennerley, M. P., Micheli, P., Martinez, V., Mason, S., Marr, B., Gray, D., & Neely, A. D. (2007). Towards a definition of a business performance measurement system. International Journal of Operations & Production Management, 27(8), 784-801. http://dx.doi.org/10.1108/01443570710763778
  » http://dx.doi.org/10.1108/01443570710763778
• Franco-Santos, M., Lucianetti, L., & Bourne, M. (2012). Contemporary performance measurement systems: A review of their consequences and a framework for research. Management Accounting Research, 23(2), 79-119. http://dx.doi.org/10.1016/j.mar.2012.04.001
  » http://dx.doi.org/10.1016/j.mar.2012.04.001
• Gunasekaran, A., & Ngai, E. W. T. (2012). The future of operations management: an outlook and analysis. International Journal of Production Economics, 135(2), 687-701.
• Hair, J. F., Jr., Black, W., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2009). Multivariate data analysis (7th ed.). Harlow: Pearson.
• Hayes, R., & Pisano, G. (1994). Beyond world-class: the new manufacturing strategy. Harvard Business Review, 72(1), 77-84.
• Hayes, R., & Upton, D. (1998). Operations-based strategy. California Management Review, 40(4), 8-25. http://dx.doi.org/10.2307/41165962
  » http://dx.doi.org/10.2307/41165962
• Hayes, R., & Wheelwright, S. (1984). Restoring our competitive edge: competing through manufacturing New York: John Wiley & Sons.
• Hudson, M., Smart, A., & Bourne, M. (2001). Theory and practice in SME: performance measurement systems. International Journal of Operations & Production Management, 21(8), 1096-1115. http://dx.doi.org/10.1108/EUM0000000005587
  » http://dx.doi.org/10.1108/EUM0000000005587
• Kaplan, R. S., & Norton, D. P. (1996a). The balanced scorecard Boston: Harvard Business Press.
• Kaplan, R. S., & Norton, D. P. (1996b). Translating strategy into action: the balanced scorecard Boston: Harvard Business Press.
• Kennerley, M., & Neely, A. (2002). A framework of the factor affecting the evolution of performance measurement systems. International Journal of Operations & Production Management, 22(11), 1222-1245. http://dx.doi.org/10.1108/01443570210450293
  » http://dx.doi.org/10.1108/01443570210450293
• Keong Choong, K. (2014). The fundamentals of performance measurement systems. International Journal of Productivity and Performance Management, 63(7), 879-922. http://dx.doi.org/10.1108/IJPPM-01-2013-0015
  » http://dx.doi.org/10.1108/IJPPM-01-2013-0015
• Koufteros, X., Verghese, A. J., & Lucianetti, L. (2014). The effect of performance measurement systems on firm performance: a cross-sectional and a longitudinal study. Journal of Operations Management, 32(6), 313-336. http://dx.doi.org/10.1016/j.jom.2014.06.003
  » http://dx.doi.org/10.1016/j.jom.2014.06.003
• Lebas, M. J. (1995). Performance measurement and performance management. International Journal of Production Economics, 41(23-25), 23-35. http://dx.doi.org/10.1016/0925-5273(95)00081-X
  » http://dx.doi.org/10.1016/0925-5273(95)00081-X
• Leong, G., Snyder, D., & Ward, P. (1990). Research in the process and content of manufacturing strategy. Omega, 18(2), 109-122. http://dx.doi.org/10.1016/0305-0483(90)90058-H
  » http://dx.doi.org/10.1016/0305-0483(90)90058-H
• Malavski, O. S., Lima, E. P., & Costa, S. E. G. (2010). Modelo para a mensuração do capital intelectual: uma abordagem fundamentada em recursos. Production, 20(3), 439-454. http://dx.doi.org/10.1590/S0103-65132010005000050
  » http://dx.doi.org/10.1590/S0103-65132010005000050
• Martins, R. A. (2012). Abordagens quantitativa e qualitativa. In P. A. C. Miguel (Org.), Metodologia de pesquisa em engenharia de produção e gestão de operações (2. ed., pp. 47-63). Rio de Janeiro: Elsevier/ABEPRO.
• Medori, D., & Steeple, D. (2000). A framework for auditing and enhancing performance measurement systems. International Journal of Operations & Production Management, 20(5), 520-533. http://dx.doi.org/10.1108/01443570010318896
  » http://dx.doi.org/10.1108/01443570010318896
• Melnyk, S. A., Bititci, U., Platts, K., Tobias, J., & Andersen, B. (2014). Is performance measurement and management fit for the future? Management Accounting Research, 25(2), 173-186. http://dx.doi.org/10.1016/j.mar.2013.07.007
  » http://dx.doi.org/10.1016/j.mar.2013.07.007
• Melnyk, S. A., Stewart, D. M., & Swink, M. (2004). Metrics and performance measurement in operations management: dealing with the metrics maze. Journal of Operations Management, 22(3), 209-218. http://dx.doi.org/10.1016/j.jom.2004.01.004
  » http://dx.doi.org/10.1016/j.jom.2004.01.004
• Micheli, P., & Mari, L. (2014). The theory and practice of performance measurement. Management Accounting Research, 25(2), 147-156. http://dx.doi.org/10.1016/j.mar.2013.07.005
  » http://dx.doi.org/10.1016/j.mar.2013.07.005
• Mura, M., Longo, M., Micheli, P., & Bolzani, D. (2018). The evolution of sustainability measurement research. International Journal of Management Reviews, 20(1), 661-695. http://dx.doi.org/10.1111/ijmr.12179
  » http://dx.doi.org/10.1111/ijmr.12179
• Neely, A. (1998). Measuring business performance London: The Economist Books.
• Neely, A., Gregory, M., & Platts, K. (2005). Performance measurement system design - a literature review and research agenda. International Journal of Operations & Production Management, 15(4), 80-116. http://dx.doi.org/10.1108/01443579510083622
  » http://dx.doi.org/10.1108/01443579510083622
• Nudurupati, S. S., Bititci, U. S., Kumar, V., & Chan, F. T. S. (2011). State of the art literature review on performance measurement. Computers & Industrial Engineering, 60(2), 279-290. http://dx.doi.org/10.1016/j.cie.2010.11.010
  » http://dx.doi.org/10.1016/j.cie.2010.11.010
• Nudurupati, S. S., Tebboune, S., & Hardman, J. (2016). Contemporary performance measurement and management (PMM) in digital economies. Production Planning and Control, 27(3), 226-235. http://dx.doi.org/10.1080/09537287.2015.1092611
  » http://dx.doi.org/10.1080/09537287.2015.1092611
• O’Mara, C. E., Hyland, P. W., & Chapman, R. L. (1998). Performance measurement and strategic change. Managing Service Quality, 8(3), 179-182.
• Okoshi, C. Y., Pinheiro de Lima, E., & Gouvea da Costa, S. E. (2019). Performance cause and effect studies: analyzing high performance manufacturing companies. International Journal of Production Economics, 210, 27-41. http://dx.doi.org/10.1016/j.ijpe.2019.01.003
  » http://dx.doi.org/10.1016/j.ijpe.2019.01.003
• Pasqualini Blass, A., Costa, S. E. G., Lima, E. P., & Borges, L. A. (2017). Measuring environmental performance in hospitals: a practical approach. Journal of Cleaner Production, 142(1), 279-289. http://dx.doi.org/10.1016/j.jclepro.2016.07.213
  » http://dx.doi.org/10.1016/j.jclepro.2016.07.213
• Pernot, E., & Roodhooft, F. (2014). The impact of inter-organizational management control systems on performance: a retrospective case study of an automotive supplier relationship. International Journal of Production Economics, 158(1), 156-170. http://dx.doi.org/10.1016/j.ijpe.2014.07.029
  » http://dx.doi.org/10.1016/j.ijpe.2014.07.029
• Pinheiro de Lima, E., Gouvea da Costa, S. E., & Angelis, J. J. (2008). The strategic management of operations system performance. International Journal of Business Performance Management, 10(1), 108-132. http://dx.doi.org/10.1504/IJBPM.2008.015924
  » http://dx.doi.org/10.1504/IJBPM.2008.015924
• Pinheiro de Lima, E., Gouvea da Costa, S. E., & Angelis, J. J. (2009). Strategic performance measurement systems: A discussion about their roles. Measuring Business Excellence 13(3), 39-48. http://dx.doi.org/10.1108/13683040910984310
  » http://dx.doi.org/10.1108/13683040910984310
• Pinheiro de Lima, E., Gouvea da Costa, S. E., Angelis, J. J., & Munik, J. (2013). Performance measurement systems: a consensual analysis of their roles. International Journal of Production Economics, 146(2), 524-542. http://dx.doi.org/10.1016/j.ijpe.2012.05.007
  » http://dx.doi.org/10.1016/j.ijpe.2012.05.007
• Platts, K., & Gregory, M. (1990). Manufacturing audit in the process of strategy formulation. International Journal of Operations & Production Management, 10(9), 5-24. http://dx.doi.org/10.1108/EUM0000000001264
  » http://dx.doi.org/10.1108/EUM0000000001264
• Platts, K., Mills, J., Bourne, M., Neely, A., Richards, H., & Gregory, M. (1998). Testing manufacturing strategy formulation processes. International Journal of Production Economics, 56-57(1), 517-523. http://dx.doi.org/10.1016/S0925-5273(97)00134-5
  » http://dx.doi.org/10.1016/S0925-5273(97)00134-5
• Pohlman, M. C. (2007). Análise de conglomerados. In L. J. Corrar, E. Paulo & J. M. Dias Filho (Coords.), Análise multivariada para os cursos de Administração, Ciências Contábeis e Economia São Paulo: Atlas.
• Prahalad, C. K., & Hamel, G. (1990). The core competence of the corporation. Harvard Business Review, 68(3), 79-91.
• Simons, R., Davila, A., & Kaplan, R. S. (1999). Performance measurement & control systems for implementing strategy: text & cases Englewoods Cliffs: Prentice Hall.
• Skinner, W. (1969). Manufacturing – missing link in corporate strategy. Harvard Business Review, 47(3), 136-145.
• Skinner, W. (1974). The focused factory: new approach to managing manufacturing sees our productivity crisis as the problem of ‘how to compete’. Harvard Business Review, 52(3), 113-121.
• Slack, N., & Brandon-Jones, A. (2019). Operations management (9th ed.). Harlow: Pearson.
• Slack, N., & Lewis, M. (2017). Operations strategy (5th ed.). Pearson: Harlow.
• Taylor, A., & Taylor, M. (2009). Operations management research: contemporary themes, trends and potential future directions. International Journal of Operations & Production Management, 29(12), 1316-1340. http://dx.doi.org/10.1108/01443570911006018
  » http://dx.doi.org/10.1108/01443570911006018
• Veiga, G. L., Pinheiro de Lima, E., Van Aken, E., & Gouvea da Costa, S. E. (2019). Efficiency frontier identification on the context of operations strategy - A study on representative constructs and variables. Procedia Manufacturing, 39(1), 745-755. http://dx.doi.org/10.1016/j.promfg.2020.01.436
  » http://dx.doi.org/10.1016/j.promfg.2020.01.436
• Wheelwright, S., & Hayes, R. (1985). Competing through manufacturing. Harvard Business Review, 63(1), 99-109.
• Wieland, U., Fischer, M., Pfitzner, M., & Hilbert, A. (2015). Process performance measurement system: towards a customer-oriented solution. Business Process Management Journal, 21(2), 312-331. http://dx.doi.org/10.1108/BPMJ-04-2014-0032
  » http://dx.doi.org/10.1108/BPMJ-04-2014-0032

Publication Dates

• Publication in this collection
  20 Nov 2020
• Date of issue
  2020

History

• Received
  27 May 2018
• Accepted
  11 Dec 2019