Acessibilidade / Reportar erro

Performance evaluation in assets management with the AHP

Abstract

The main objective of this article is to develop a transparent and efficient model, using the AHP (Analytic Hierarchy Process) method as a basis and applying the software SuperDecisions, which makes it possible to evaluate the performance of employees, aligning the individual objectives defined and the competences required for the organisational mission. The methodology used was of an investigative and exploratory nature, due to the need to increase knowledge on the use of the AHP in the process of classifying employee performance. Starting from a high performance culture, it is possible to highlight the employees who present the best performance and improve the remuneration policies, as well as invest in the training of the skills lacking in a clear and assertive manner. The results found in this research prove the viability of the respective application, which is directly related to the meeting of targets and, consequently,to sustaining competitiveness and increasing added value for shareholders.

human resources; performance evaluation; multicriteria decision aiding


Performance evaluation in assets management with the AHP

Luiz Flávio Autran Monteiro Gomes* * Corresponding authorIbmec/RJ, Av. Presidente Wilson, 118, Sala 1110, Centro, 20030-020 Rio de Janeiro, RJ, Brazil.E-mails: autran@ibmecrj.br / rmarques@csp.org.br ; Renata Marques de Andrade

ABSTRACT

The main objective of this article is to develop a transparent and efficient model, using the AHP (Analytic Hierarchy Process) method as a basis and applying the software SuperDecisions, which makes it possible to evaluate the performance of employees, aligning the individual objectives defined and the competences required for the organisational mission. The methodology used was of an investigative and exploratory nature, due to the need to increase knowledge on the use of the AHP in the process of classifying employee performance. Starting from a high performance culture, it is possible to highlight the employees who present the best performance and improve the remuneration policies, as well as invest in the training of the skills lacking in a clear and assertive manner. The results found in this research prove the viability of the respective application, which is directly related to the meeting of targets and, consequently,to sustaining competitiveness and increasing added value for shareholders.

Keywords: human resources, performance evaluation, multicriteria decision aiding.

1 INTRODUCTION

Performance evaluation is related to two basic objectives: rewarding employees for targets met and identifying the proposals not achieved so that an action plan can be defined and thus all objectives achieved in the future (Islam & Rasad, 2006). Individual performance evaluation is based on criteria specific to the performance of the work which must be aligned with the projects with which each employee is involved. Therefore, depending on the project, the objectives, the mission and the organisational vision and the criteria for the evaluation will have distinct weights (Cheng & Li, 2006). In this way, the weights must be based on the function carried out by the employee, as the objectives and the competences are directly related, with a greater or smaller contribution, to achieving business success. More important than developing the organisational strategy is to execute it in the light of the impact of the activities on the results of the organisation. A human resources management strategy aligned to organisational strategy guarantees an increase in productivity and profitability, consequently increasing shareholder value. Only by the measuring and monitoring of the impact of individuals in the success of the organisational strategy is it possible to evaluate the value added by investments carried out in Human Resources (Beatty, Becker & Huselid, 2005).

The purpose of this work is to apply a Multicriteria Decision Aiding method to the process of functional performance evaluation with the aim of increasing the efficiency and transparency of the process. The method selected was the AHP (Analytic Hierarchy Process), using the software SuperDecisions. Thus, starting from the stratification of the organisational mission into objectives and behavioural and technical skills – in this way, considering both qualitative and quantitative aspects – the employees are evaluated based on their individual contribution to the execution of the strategy and, therefore, to the realisation of the mission. In summary, the use of the methodology was designed, through performance evaluation, to align the competences and behaviours to the organisational strategy, developing perspectives of how people can contribute to its execution. Through a policy of high performance, the positions which contribute directly to achieving the strategic objectives are highlighted, as well as the employees who present the best performance. The AHP method has been widely applied to different objectives in political,economic and social areas. In the area of people management, it has been widely used in the evaluation and selection of candidates, based on the definition of and scoring in competencesand skills essential to the performance of the function (Tett et al., 2000; Swiercz & Ezzedeen,2001; Thomaz, 2006). It is also worthwhile highlighting the application of the methodology as an indicator selection tool for the measurement of organisational performance (Kimura & Suen, 2003). Few works are to be found in relation to the use of a Multicriteria Decision Aiding methodology as an employee performance evaluation tool, among them, the works of Islam & Rasad (2006) and Cheng & Li (2006). In the first research, the AHP was used to map individual performance, based on the quantification of the criteria and subcriteria considered. In the second study, the method used was the Analytic Network Process (ANP). A number of other articles are available that deal with the use of the AHP in performance evaluation (Liu et al., 2005; Hunjak & Jakovcevic, 2001).

2 THEORETICAL FRAMEWORK

2.1 Evaluation of team and individual performances

Organisations are living organisms (Bertalanffy, 1977), immersed in a larger system, which are made up of subsystems, that is, parts which relate to each other. In this way, organisations are open systems which communicate with the environment (Katz & Kahn, 1970), and, for this reason, the environment must be monitored so that it is possible to interfere in accordance with the interests of the organisation, or at least adapt to the conditions offered by the environment (Lawrence & Lorsch, 1973). Organisations are planned systems which are maintained by connections which need control to keep their parts working (Katz & Kahn, 1970). In summary, the interaction between the structural and human dimensions of organisations and the influence of the environment means that organisations are composed of interactive and interdependent subsystems (Bowditch & Buono, 2002). Tools are developed, based on the organisational needs, with the purpose of evaluating performance, developing human resources and improving performance at work, as well as controlling the quality of the activities developed. In this way,evaluation presupposes a comparison of the performance achieved with that which was expected and the correction of occasional deviations so as to reach the planned objectives (Goodale, 1992;Lucena, 1977; Aluri & Reichel, 1994).

Performance evaluation is a formal system which measures behaviour and results to evaluate the productivity of an employee. The better the performance, the more the employee, the company and society benefit (Shaout & Al-Shammari, 1998). This system evaluates employees who carry out a determined function for a specific period of time (Deadrick & Gardner, 2000). Before making the evaluation, itself, the evaluator must make the effort to rank the performance of all the employees from best to worst performance (Falcome, 2007).

As well as the differentiation in performance, the positions must also be weighted according to the contribution to achieving the main objective and to the execution of the strategy. In this way,the contribution must be directly related to the lines of work and to the financial return generated for the organisation. The differentiation is fundamental to the construction of a culture groundedin high performance (Larry, Charan & Burck, 2002; Beatty, Becker & Huselid, 2005). Performance management therefore involves planning, monitoring and evaluation. In the first stage,the objectives and targets are stipulated, as well as the indicators and the level of performance desired, on which the performance achieved will be evaluated. The objective of the monitoring stage is to identify and correct deviations and implement solutions for the corrections of faults, in order to reach the objectives stipulated. The last stage is the evaluation of functional performance itself (Guimarães, Nader & Ramagem, 1998).

For Durand (1998), competence is the interdependence of knowledge, skills and attitudes necessary for the execution of the targets. For Dutra, Hipólito & Silva (1998), competence is thecapacity to generate results from the organisational targets stipulated. Ropé & Tanguy (1997)summarise the discussion and state that whatever the competence to be analysed, it cannot be understood separately from the action. In this way, competence materialises only when there is assimilation of the knowledge by means of individual and collective learning, integration of skills and adoption of attitudes, in the specific organisational context. In this aspect, the strategic management of human resources contributes to the development of competences which make it possible to sustain competitive advantage (Taylor, Beechler & Napier, 1996). Competences are divided into individual and organisational. Individual competences are related to individual skills learnt throughout one's life and which collaborate towards the success of the whole organisation, such as, for example: behavioural skills, technical knowledge, formal education and previous professional experience (Kennedy & Dresser, 2005). Organisational competences generate differentiated value, as they consider the organisational mission, the vision, the values and the organisational strategy as being recognised by clients as differentials, which would be imitated with difficulty by the competition (Prahalad & Hamel, 1990).

The targets and objectives which must be reached to realise the strategy, as well as the organisational competences necessary, are defined from the establishing of the mission and organisational vision. Afterwards, the employees are evaluated in order to establish the internal competences available and the gap existing between ideal competences and real competences (Ienaga, 1998).Indicators and measurements have the aim of "[...] pulling people in the direction of the vision"(Kaplan & Norton, 2005, p. 180), provided that they are measured and monitored (Kaplan &Norton, 2005). According to Kennedy & Dresser (2005) different businesses require different competences and these, in turn, are more observable in the results produced and are evaluated objectively with difficulty. The success of the organisation is intrinsically related to the performance of the individual competences, based on the execution of the organisational strategies.In addition, the focus on employee appraisal, based on the organisational strategy, promotes advances in divulging the organisational values, as well as clarifying the understanding of the organisation concerning the high performance necessary for individual career development.

The performance evaluation system is complex and can be associated both with general individual performance and a variety of criteria, whether qualitative or quantitative (Meyer et al.,1989). Lack of success in the evaluation system lies in a sole qualitative focus on skills, whichis often used as a tool of benevolence and complacency (Falcome, 2007). Performance evaluation is indeed one among the ten ways which favour the creation and maintaining of competitive advantage (Longenecker & Fink, 1999). On the other hand, a precarious system of performance evaluation can provoke low self-esteem, a decrease in productivity and enthusiasm and,consequently, little contribution towards achieving the organisational mission (Somerick, 1993).When the system is seen as unjust, the results of performance evaluation are not accepted by the employees, making the reception, feedback and improvement of the process impossible. In this way, the effective management of a system of performance evaluation is a tool for measuring and improving productivity (Islam & Rasad, 2006). Therefore, in this article the evaluation is given by the association of competences with the objectives, understanding that the evaluation of this union translates the capacity to transform knowledge, skills and attitudes (competence)into results.

2.2 The decision making process

In part our choices determine who we are and whether we will be successful in our attitudes.It is starting from our decisions that we deal with facts and we learn, though the successes or frustrations experienced in the face of these choices, to live with the situations that life presents to us (Hammond, Keeney & Raiffa, 2004). Decision support methods do not promise to point out a single governing truth to the detriment of all others, but, from the decision maker's (or decision making group's), preferences to support the decision making process and recommend actions in multiple objectives problems, indefinitions, uncertainties, multidisciplinarities and the weighting of non-quantifiable criteria among other questions. The integration of the quality of the information, together with the analytical treatment of the information, is the materialisation of Multicriteria Decision Aiding.

The stages of the decision making process are the compiling of the information, modelling,selection and revision. In the first stage, the objective is to recognise the problem and to collect the information and needs, while in the second, from the information collated in the first stage, the objective is to formulate and define the problem. In the third stage, the objective is to model the preferences with the aim of constructing the analytical model of the problem. The objective of the last stage is to perform the analysis itself of the alternatives according to the model constructed (Gomes, Araya & Carignano, 2004). The Multicriteria Decision Aiding process is represented by a decision making problem, which aims to consider simultaneous objectives through different functions. This process can be mathematically represented by: Max F(x), where x X and x is the vector [x1, x2,..., xn] of n decision variables, called the set of alternatives. The term F(x) is the vector of [ f1(x), f2(x), . . . , fp(x)] of p objective functions. To the extent that the management and strategic goals are threatened by risks and uncertainties, the use of a scientific model which considers uncertainty is essential, understanding that, more and more, the reduction of losses added to optimisation of the results increase companies' competitive margin (Belton & Stewart, 2002).

The objective of this work is to use the Multicriteria Decision Aiding method AHP, in order to analyse a γ (i.e. ranking) type of problem as defined by Roy & Bouyssou (1993). In this way,the method will offer, through multiple binary comparisons, the ranking of the performance of the employees (set of alternatives) which make up the sample in question, from the criteria and attributes collated together with the decision maker.

2.2.1 The classic AHP method

According to Thomas Saaty (1991), the creator of the AHP, the theory of this multicriteria method reflects the functioning of the human mind, that is, the natural way in which the mind deals with a large quantity of information, characteristic of complex situations. To facilitate understanding, the mind sorts elements with common properties into groups. The repetition of the model permits the aggregation of new elements into subgroups, at different levels of the system.This dynamic permits the structuring of a hierarchy, where the maximum level is composed of a single element, which is the objective of the decision making process. The purpose of the stratification is to verify the weight (w1,w2,...,wn) with which the elements (c1, c2,..., cn)of the level immediately below influence the level above and, consequently, the impact that each of these elements exercises on the main objective. The weights are attributed through a sequence of pair comparisons of dominance of the lowest factors in relation to the objective (Saaty, 1977).The model embedded in the use of the AHP must express the reality, including and measuring tangible and intangible factors in decision making, that is, quantitative and qualitative elements,being based on a ratio scale for the measurement (1 to 9) and on the pair comparisons of the elements derived from the priorities of each of the alternatives selected (Saaty, 1977; Meade& Sarkis, 1999; Dikmen, Birgonul & Sedat, 2007). These comparisons need to be consistent,going far beyond preference transitivity. Thus, being consistent not only means respecting the association that if a is preferable to b and b is preferable to c, then a is preferable to c, but how the real intensity is expressed throughout the system through pair comparisons. Having cardinal consistency in the force of preference is to affirm that if a is twice preferable to b and b three times preferable to c, then a is six times preferable to c (Saaty, 1991).

The scale of priorities is composed of the interval from 1 to 9. According to Saaty, consistency is the capacity, through a determined number of data, to logically deduce the others (Saaty, 2002).In this way, values are assigned to express the dominance of one element over another, with the respective reciprocals conferred to the expression of preference of the inverse relation. When an element is compared to itself, it is given the value 1, indicating that it deals with the same degree of preference (Saaty, 1977; 1991). The decision making process will therefore reflect the preferences of the decision maker, as it ratifies individual perceptions in relation to the level of dominance of the elements of a group or subgroup in the decision making context.

It is not important how inconsistent the decision maker is in specific comparisons but how much the consistency is violated in the general study of the problem. The consistency is expressed by the consistency index (CI), the result of the subtraction of the maximum eigenvalue (λmax) by the number of elements considered in the matrix (n) and divided by the subtraction of n minus one. Thus, the mathematical expression is represented by:

For the consistency to be acceptable, the ratio between the CI and the RI (Random Index) must be less than 0.10. The mean values of the random indices were determined by an experiment carried out at the National Laboratory of Oak Ridge, based on a sample size of 100 for matrices of order 1-15. Thus, the value attributed depends on the order of the matrix, according to Table 1 (Saaty, 1977).

As a result of its psychological and mathematical basis, and because it considers both quantitative and qualitative elements, the AHP is currently one of the Multicriteria Decision Aiding methods most widely used in Brazil and the rest of the world. The methodology is structured in the software SuperDecisions, Decision Lens and Expert Choice, being used to model problems related to the allocation of resources, the prioritisation of projects, the appraisal of candidates,marketing strategy, cost/benefit analysis, the selection of suppliers and credit analysis among others. This work will apply the tool to rank the performance of employees, based on the impact of the individual contribution to the realisation of the organisational strategy. The software used in this study will be SuperDecisions (Saaty, 2002).

Neverthless one cannot ignore that the AHP has been the object of serious criticism since its appearance in the litterature in the eighties. Some of the main critics follow (Goodwin & Wright,2000): (i) Conversion from a verbal to a numerical scale: decision agents using verbal comparisons will have their judgements automaticaly converted to a numerical scale; however, the correspondence between these two scales is generally based on non tested hypotheses. (ii) Using the scale from 1 to 9 may cause inconsistencies; for instance, if A is considered 5 times more important then B and B is 6 times more important than C, then C, in order not to violate the principle of strict transitivity, should be 30 times more important than c, which is impossible. This criticism is also mentioned by Barzilai (2001), when this author points out the limited flexibility of the inputs from every decision agent. (iii) Ambiguity in the answers to the questions posed to decision agents: weights are obtained with no reference to the scales on which attributes are measured; this may lead to different understandigs of the questions, which is a potential source of errors. Lootsma (1990) pointed out the difficulties faced by decision agents in the selection of one among different verbal qualifications in order to express their preferences and choose between any two alternatives, particularly when performamces are expressed in physical or monetary values. (iv) New alternatives may revert the initial ranking of alternatives: this issue was raised by various authors such as Belton & Gear (1982), Dyer & Ravinder (1983) and Lootsma (1990). Saaty & Vargas (1984) answered to that criticism by claiming the legitimity of rank reversal, which was the object of a surrejoinder by Belton & Gear (1990). Having in mind that rank reversal results from the way weights are normalized in the AHP, Dyer (1990) then suggested a solution to the rank reversal problem in the AHP. (v) The number of pairwise comparisons required may be considered too large which may lead to errors by fatigue. (vi) The axioms of the method: Dyer (1990) pointed out that the axioms of the AHP are not founded on descriptions of rational behavior that can be tested; Harker & Vargas (1987) have presented strong objections to this sixth criticism though. However, it was Bana e Costa & Vansnick (2001) that presented one of the most bruising criticisms to the AHP. These two authors identified a problem that occurs in the computation of the vector of priorities, particularly in the scales derived from the method,from a reciprocal positive matrix filled by questioning the decision agent. That problem has implications in the way priorities are quantified and not in the order of priorities. A quite serious aspect also raised by Bana e Costa & Vansnick (2001) is that the coefficient of inconsistency proposed by Saaty is not able to detect such flaw of the AHP.

Even taking into consideration that the AHP has been the object of intense scientific debates in the Operations Research literature, the value of such method as a tool for building a requisite model (Phillips, 1982; 1983) for a complex, multicriteria decision problem is irrefutable. In other words, the use of the AHP can provide structuring the decision problem as a hierarchy of objectives on which a consensus can be reached and based on which different decision agents can interact by eliciting their value judgements. This was in fact the rationale for using the AHP in the performance evaluation study presented in this article. It can also be added that since performance evaluation in the context of that study was prone to be tackled through a hierarchy of criteria made the selection of the AHP natural as the multicriteria analytical method to be used. For those reasons the AHP was selected among various other Multicriteria Decision Aiding analytical methods.

3 METHODOLOGY

The methodological approach used was action research, as the purpose was not only to understand or describe a process but transform it. Action Research began with Kurt Lewin, in the 1940s, by aggregating, through experimental research, criteria and objectives to problems in social sciences (Eden & Huxham, 2001). In this type of research, the agent and researcher communicate and cooperate towards a perfect understanding of the reality in which they are inserted,and seek the solution, producing and structuring knowledge and delineating actions (Thiollent,1997). This being the case, the use of action research is justified in the search for reliable results and actions committed to the idealised objectives and results, in order to provoke change (Lima,2005). The difference between action research and a case study is the relationship developed between the researcher and the people in the organisation participating in the project (Bryman,1989). While the first depends on the objective and the context in which it is immersed, the second originates from a single phenomenon which is revealed from multiple documents (Thiollent,1997; Voss, Tsikritsis & Frolich, 2002). The collection of data in action research is carried out through collective and individual interviews, field work observation, database research etc., with the groups and individuals chosen based on selective or statistical criteria (Thiollent, 1997). As a result of the characteristics described, action research, together with empirical research, were the methodologies used to investigate this question: empirical, because the problem was studied in an assets management company administered by a majority Canadian shareholder, and the holder of 90 billion dollars in assets around the world.

In relation to the means of investigation, the research involved both field work and bibliographical research. It is considered field work as it was necessary to do an initial market research study to understand the position occupied by the company, as well as the analysis of the internal Human Resources policies and documents published, both for the employees and for the (future) shareholders, in Brazil and abroad. Afterwards, the documents related to the evaluation performance process itself were studied, both from the Brazilian branch, called company F in this work, and in the transnational company, called company M, as well as the policies, forms,evaluations and target programs. Lastly, after the documentary analysis and the interviews with the financial director and the Controlling department manager, the problem was modelled, discussed and adjusted for validation and application of the tool. In relation to the bibliographical research, it was necessary to research material published in different media.

The structuring of the research is represented in Figure 1, where the decision process is intrinsically linked to the strategy, in other words, to the mission and the vision for the future, which,in turn, are also influenced by the environment in which the company is inserted, and by the demands of the majority shareholders. In this way, it is from the construction of the current ideal scenario X, that the competences are defined, together with the objectives, making it possible to create the hierarchisation of the model and the weighting of the degrees of dominance of the attributes and the alternatives, through the use of the SuperDecisions software.


4 MULTICRITERIA ANALYSIS WITH THE AHP

The research structure was carried out based on an explanatory study (Cooper & Schindler,2003), as it was necessary to collect information, with the purpose of increasing knowledge concerning the performance evaluation process and the use of the AHP method as a tool for the ranking of the performance of the employees, in relation to the impact of the individual's contribution to the execution of the strategy. The intention was to investigate the hypothesis that the use of a Multicriteria Decision Aiding method increases transparency, aligning the performance evaluation process to organisational strategy.

This study was developed in company F, the Brazilian branch of a transnational company, acting in the assets management market – whose shareholder is the Canadian company M – the holder of 100% of the shares, with, approximately, 90 billion dollars in assets in the world and open capital in the stock exchanges of New York, Toronto and Amsterdam. With a 100 year history of investing in the country, the economic group in Brazil currently has more than 4,750 employees and has one of the largest investment platforms in the country, with more than 11 billion reais under its management, including its own and institutional clients' assets, investing in the property, renewable energy, agriculture, forestry and infrastructure sectors. It is worth recording that the data cited here was collected in February 2009.

Approximately two years ago, company F radically altered its strategy, ceasing to act solely as controller of the companies in the country and starting to work in assets management through the use of investment funds. In this new structure, as well as the company beginning to act as a fund investor company, resources are captured from institutional investors, mainly foreign ones,interested in investing in assets in Brazil, giving the fund the leverage to acquire and manage these assets. The operational companies, beforehand controlled, are then sold to the funds created,becoming their operational 'arms'. The main differential of the company's philosophy in the market in question is not only to be the 'structurer', but also direct investor and manager of investment funds. In this way, the assets which compose the fund belong to the operational companies, which, in turn, are managed by company F. If the company was only a management company, it would receive the management rate levied as reward for services provided, but, as it is also an investor company, it is in its real interest to increase the profit margin of each of the investment funds. From this alteration and the much closer intervention of company M, it became necessary to collect data not only of the main concepts concerning company F, but also clarification of the understanding of the policy and economic position expected by company M.In addition, it was necessary to understand how the performance evaluation processes concerning the mapping of the competences are aligned to the mission, vision and values, making company M one of the largest asset management companies in the world.

Data collection was carried out through accessing documents as well as non-structured interviews with employees in the Human Resources area of the Head Office in Canada and the branch in the United States. Later, the same data collection was carried out in company F, through accessing documents and non-structured interviews with the president of company F, with the presidents of each of the investment funds, as well as the financial, administrative and technological management committee and the manager of the Controlling area. The exploratory technique used was an experience survey, with the aim of seeking important information for the development of the present research, through individual experience and knowledge, with the instrument flexible enough for new explorations to be made during the interview (Cooper & Schindler,2003). As company F did not possess a defined mission and vision of the future, the construction of the organisational strategy constituted the first part of the work. Once in possession of this information, and understanding the new strategic position assumed, the technical and behavioural competences were highlighted and validated.

The ethical questions on the research proposal were approached though restrictions stipulated by the organisation, such as not divulging the name of the Brazilian company nor the shareholding company, due to the confidential nature of some data involved in the research. Another action recommended was the anonymity of the employees evaluated and the evaluator, protecting them from any exposure and discrimination that they could come to suffer.

For Straight (2000), the first stage, prior to the selection of the performance measurements, is to define the objectives. If this has not been obeyed, the measurements will not be valid, as they will not be related to the organisation. In addition, the team must be involved in the definition of the measurements, in order to develop team commitment and reach them. In this way, in the research in question, the hierarchy must be constructed from top to bottom, as it is from the definition of the principal objective in the hierarchy, in the stated problem represented by the organisational mission, that the other criteria and subcriteria can be defined. The employees will show commitment to reach the organisational goals to the extent that the evaluation is continuous and the objectives are established and revised by the evaluator together with those evaluated.The objectives were mapped from the organisational mission and the competences evaluated were defined from the mission described. The competences mapped in the current performance evaluation forms of company F were also considered.

After the data collection above was carried out, the competences were corrected and indicators created for the evaluation of behavioural skills, as it was necessary to reinforce what should be understood as, for example, effective communication, as presented in Table 2. In this way,the intention was to reduce the margin of error in relation to that which should be understood about the skill referred to. In addition to behavioural skills, technical and scientific skills were also considered. In this way, all behavioural perspectives were considered, as behaviour is the materialisation of competence linked to knowledge, skills and attitudes for the execution of goals,generating results (Durand, 1998; Dutra; Hipólito & Silva, 1998).

The judgement process and high level validation was of great importance for the identification of the performance priorities and the refinement of the analysis, as well as for the definition of the criteria considered inside each of the subgroups (competences, organisational objectives,behavioural skills, technical and scientific skills and behavioural indicators). Having collected and validated all the aspects to be considered, the problem was presented through the software SuperDecisions.

The main objective, expressed by the organisational mission, is located at the first level of the hierarchy and was decomposed into Objectives and Competences. In turn, the competences weredivided into subcriteria at the third level, classed as Technical and Scientific and Behavioural Skills, while the objectives were decomposed into Objective 1, Objective 2, Objective 3, Objective 4 and Objective 5. Behavioural skills were again divided into behavioural indicators,until the lowest level of the hierarchy was reached, represented by the alternatives which, in the study in question, are the employees evaluated, according to Figure 2. Considering the quantity of information involved, it is difficult to state immediately which employee demonstrates the best performance or which most contributed to the realisation of the strategy. Although some dominated in one specific objective, they are dominated in other competences for example. In this way, all the alternatives were evaluated according to behavioural indicators, technical and scientific skills and the area objectives taken from the organisational mission. Thus it was possible to rank the evaluated performances according to the contribution to the success of the organisational mission.


The research sample was composed of 12 employees from the same branch, including the controlling, accounts and accounts payable areas. Having hierarchised the problem, the decision maker, represented by the direct manager, evaluated, from his/her own perception, the degree of importance of each element in relation to the element immediately above. Firstly, the importances were attributed related to each of the subcriteria at the same level and, afterwards,the relative importances of the elements belonging to each of these subcriteria were evaluated.This attribution of priorities has the purpose of defining the weights for all the categories decomposed from the main objective of the hierarchy, as well as the elements which form each of the (sub)criteria. It is worth highlighting that, after the analysis of the calculations, the behavioural skills are moderately more important than the technical skills (Figs. 3 and 4). In this way, thecompetences responsible for differentiated performance are those which can be observed, tested and modified, in other words, the technical skills, while the behavioural competences are those of difficult access. The small dominance of the behavioural aspects over the technical ones is due to the fact that, as they are difficult to monitor and modify, the organisation has difficulty in managing them, in contrast to the technical skills, which can be improved through training and development. In this way, the organisation expects to lead the employees to reflect on those elements of which only they have profound knowledge (Kennedy & Dresser, 2005).



Once all the judgements have been made, the relative weights and the inconsistency indices of each evaluation were defined, as well as the global weights which represent the weighting of each element in a general analysis. In this way, the global weight is equal to the relative weight of the perspective, multiplied by the weight of the indicator inside the perspective mentioned,according to Tables 3 and 4. After the weighting of the relative and global weights of each perspective, the pair comparison is initiated of the performance of each of the employees, relative to the behavioural and technical and scientific skills, as well as to the objective strategies for the realisation of the organisational mission.

According to Saaty (1977), the consistency index must be limited to 0.10. In this way, it is concluded that the management judgements are adequate, provided that none of the evaluations goes beyond this limit, as demonstrated in Table 5. It is worth highlighting that no judgement needed to be redone, attributing the current consistency indices to the initial judgements. If the degree of consistency went beyond the limit described, the decision maker would have to re-evaluate the relative importances attributed, as the index would indicate discrepancies in the judgements.

Having identified the relative and global weights, all the employees (alternatives) are compared in pairs, according to the objectives (1, 2, 3, 4 and 5), technical and scientific skills (technical knowledge of the legislation in vigour, mastery of the work tools, experience in the role and in similar businesses, specific scientific background in the area of actuation and level of proficiency in the English language), and behavioural indicators related to efficient communication (C1,C2, C3 and C4), focus on the business (F1, F2, F3 and F4), initiative and pro-activeness (I1,I2 and I3) and team work (T1, T2, T3 and T4), as seen in Figure 2. It falls to the manager indeed, based on the leadership exercised, to attribute the degrees of dominance between each of the indicators previously described for each of the employees, through the matrices of pair comparisons. These relative judgements refer to the evaluation of each employee – from the pair comparisons – on each of the aspects discriminated. This stage was one of the most difficult for the decision maker, in particular because of the exhaustive need for reflection, and consequently expense of time, for the respective judgements. For the construction of the present work, the judgement phase was of great importance, as it is through innumerable pair judgements that the matrix limit is constructed, resulting in the hierarchisation of the alternatives located at the last level of the hierarchy. Even though, at times, there may be a subjective tendency in the comparisons, this is reduced when comparing an employee with another from the same team.

The manager is thus "forced" to minimise subjectivity and to highlight the employee with the best performance.

Weighting the values of each one of the behavioural and technical indicators, it is possible to arrive at the final matrix of the model, as the AHP model permits the performances of each employee to be placed under a quantitative perspective, leading to more transparent and realistic results. In this way, from the hierarchisation of the strategy, we can conclude that the decreasing order of the employees who present the best performance, in other words, who most contribute to the realisation of the mission are: Employee 6, Employee 5, Employee 1, Employee 4, Employee 3, Employee 10, Employee 2, Employee 11, Employee 9, Employee 7, Employee 12and Employee 8, as presented in Figure 5. Thus, practices of variable remuneration should compensate the employees with better performance, while the training and development unit must focus on the employees and competences to be improved. In this way, the performance evaluation, according to the AHP model, provides for a more transparent and just process, as well as greater efficiency in investment in training and development. In addition, after mapping the competences, as well as the levels characterised as high performance, the Recruitment and Selection unit will certainly be more assertive in its processes.


5 CONCLUSIONS

Multicriteria analysis models can be used in problems which have the objective of selecting,ranking, classifying or describing alternatives (Roy & Bouyssou, 1993). In this way, MulticriteriaDecision Aiding tools can be used in various Human Resources subsystems such as, for example,Recruitment and Selection and Training and Development, in the evaluation of candidates based on fundamental requirements for the performance of a function or in the allocation of available resources in a determined group of employees for the improvement of a certain skill. As shown in the present work, the AHP can also be used in the performance evaluation process. Its success is due to the fact that this process considers qualitative and quantitative aspects, as well as the methodology used, being of great use in the classification of the performance of the employees evaluated (alternatives).

In the face of the change concerning the strategic positioning occupied by company F, this research provided, through the application of the AHP, the use of a performance evaluation process aligned with the organisational strategy, as well as the clarification of individual targets and the definition of the competences necessary to accomplish the vision of the organisation's future. In addition, the correction process, as well as the reinforcement of beliefs, attitudes and behaviour,though a culture which, through coherence, communicates high performance, reduces cognitive dissonance, that is, disequilibrium caused by internal conflicts resulting from the clash of thoughts, actions and values between employees and the company (Bowditch & Buono, 2002).

The model was constructed from the organisational mission and the aims and competences mapped for the realisation of the strategy, conciliating the use of the AHP. This model was elaborated from quantifiable judgements and a set of weights attributed by the decision maker who, in the present research, is represented by a single person, the manager of the Controlling,Accounts Payable and Accounts area. The great benefit was the reflection encouraged by the pair comparisons of the different objectives and requirements based on the degree of dominance.

The use of the AHP model in the employee performance evaluation process favoured transparency and efficiency, to the extent in which it scored objectives and competences based on the realisation of the organisational strategy. In addition, with the help of the SuperDecisionssoftware, it permitted the attribution of different weights to all the (sub) criteria levels, as well as the checking of the consistency of the judgements made. In relation to efficiency, based on the performance evaluation process promoting a high performance culture, policies such as posts and salaries, recruitment and selection and training and development would be more assertive,as they would emphasise behaviour which favoured the strategy and would invest time and resources in the improvement of strategic competences with lower indices. The methodology can be used in other scenarios, provided that the objectives and competences are suitable to the new reality. One important factor is the ease with which the results can be modified and new simulations can be checked in such a way that it is possible to verify how the classification of employee performance is affected.

When comparing the process with the application of the AHP and the classification obtained without the use of the Multicriteria Decision Aiding method – in other words, only by the scores attributed by the same manager in the last period, according to Figures 6 and 7 – it can be observed that the classification is distinct from that arrived at through the use of the methodology: Employee 10, Employee 6, Employee 1, Employee 9, Employee 11, Employee 3 and Employee 5. It is important to highlight that Employees 2, 4, 7, 8 and 12 do not appear in the classification without the tool, as they did not form part of the staff in the last appraisal carried out.



The inclusion and/or leaving of some members of the team could have altered the relations existing between the team, employees and manager, resulting in changes in behaviour and,consequently, in the individual performance of the employees. Furthermore, the change in the strategic positioning of company F, provoked changes in the construction of the mission and organisational vision, in this way having an influence on the objectives and measurements defined.

The new scenario imposed by the shareholding company, in conjunction with the reformulation of the targets adopted by company F and the increase/ alteration in the team formation – with the aim of meeting the current necessities – could also have influenced the distinct classifications of the performances obtained in the evaluations with and without the support of the AHP tool.If the organisational mission is altered in the future and/or the company wishes to include all the areas in an evaluation process supported by the AHP, the data collection of the competences and objectives will need to be carried out once more, as well as the construction of the hierarchy by new pair comparison judgements.

At times, the traditional performance evaluation models are based on aspects which are purely subjective and, in some cases, paternalistic. It is worth highlighting that the veracity of the performance appraisal process of employees in a team, in the majority of cases, is prejudiced not by any intentional wish of the decision maker to favour one employee over another, but as a result of the psychological traps constructed in the mind of the evaluator. If the manager does not know the dangers, he/she may become a victim of his/her own mind. With the application of the AHP, the decision maker, through pair comparisons, is led to construct a forced curve, classifying the performances of the employees in the team. In this way, eminently subjective characteristics can have their impact measured via the individual contribution towards the realisation of the organisational mission. In addition to this, through the AHP method, objectives, behavioural,technical and scientific skills are scored according to the impact on realising the strategic goal,based on an objective and quantitatively grounded mechanism.

In spite of the limitations of the multicriteria tool in relation to the definition of the fundamental scale, the model permits subjectivity in the attribution of the weights, as well as the choice between different alternatives, to be taken into consideration in the decision making, through a numerical and replicable procedure. Thus, through the use of the AHP, all the criteria and subcriteria were attributed a unique and global value which showed how these elements influence the attainment of the greatest objective of the hierarchy. Based on the success of the study Company F has decided that the AHP method should definitely be considered as the analytical tool to be utilized in their future performance evaluation processes.

ACKNOWLEDGEMENTS

The authors are grateful to the referees for their insightful comments on the first version of this paper. This work was partially supported by CNPq throught Research Project No. 310603/2009-9.

Received September 11, 2009 / Accepted May 31, 2011

  • 1. ALURI R & REICHEL M. 1994. Performance evaluation: a deadly disease? The Journal of Academic Librarianship, 20(3): 145155.
  • 2. BANA E COSTA CA & VANSNICK J-C. 2001. Une critique de base de l'approche de Saaty: mise en question de la methode de la valeur propre maximale. Cahier du LAMSADE, Université Paris-Dauphine, Paris.
  • 3. BARZILAI J. 2001. Notes on the Analytic Hierarchy Process. In: NSF Design and Manufacturing Research Conference. National Science Foundation, Tampa.
  • 4. BEATTY R, BECKER B & HUSELID MA. 2005. Scorecard para Recursos Humanos: conceitos e ferramentas para medir a contribuição das equipes. Elsevier, Rio de Janeiro.
  • 5. BELTON V & GEAR AE. 1982. On a Short-coming of Saaty's Method of Analytic Hierarchies. Omega, 11(3): 226230.
  • 6. BELTON V & GEAR AE. 1985. The Legitimacy of Rank Reversal A Comment. Omega, 13(3):143144.
  • 7. BELTON V & STEWART TJ. 2002. Multiple criteria decision analysis: an integrated approach.Kluwer Academic Press, Boston.
  • 8. BERTALANFFY LV. 1977. Teoria Geral dos Sistemas. Vozes, Petrópolis.
  • 9. BOWDITCH JL & BUONO AF. 2002. Elementos de Comportamento Organizacional. Pioneira Thomson, São Paulo.
  • 10. BRYMAN A. 1989. Research Methods and Organization Studies. Routledge, London.
  • 11. CHENG EWL & LI H. 2006. Job Performance Evaluation for Construction Companies: An Analytic Network Process Approach. Journal of Construction Engineering and Management, 132(8): 827835.
  • 12. COOPER DR & SCHINDLER PS. 2003. Métodos de Pesquisa em Administração. Bookman, Porto Alegre.
  • 13. DEADRICK DL & GARDNER DG. 2000. Performance distributions: measuring employee performance using total quality management principles. Journal of Quality Management, 4(2): 225241.
  • 14. DIKMEN I, BIRGONUL MT & SEDAT H. 2007. Using Fuzzy to rate cost overrun risk in international construction projects. International Journal of Project Management, 25(5): 494505.
  • 15. DURAND T. 1998. Forms of incompetence. In: International Conference on Competence-Based Management, 4, Norwegian School of Management, Oslo.
  • 16. DUTRA JS, HIPÓLITO JM & SILVA CM. 1998. Gesão de pessoas por competências. In: Encontro Nacional da Associação Nacional dos Programas de Pós-Graduação em Administração, 22, Anpad, Foz do Iguaçu.
  • 17. DYER JS. 1990. Remarks on the Analytic Hierarchy Process. Journal of the Institute do Management Sciences, 36(3): 249258.
  • 18. DYER JS & RAVINDER HV. 1983. Irrelevant Alternatives and the Analytic Hierarchy Process.Working Paper, The University of Texas, Austin.
  • 19. EDEN C & HUXHAM C. 2001. Pesquisa-ação no estudo das organizações. In: CALDAS M, FACHN R & FISCHER T. Handbook de estudos organizacionais. Atlas, São Paulo.
  • 20. FALCOME P. 2007. Big-Picture Performance Appraisal. HR Magazine, 52(8): 97100.
  • 21. GOMES LFAM, ARAYA MCG & CARIGNANO C. 2004. Tomada de deciãao em ceáarios complexos: introdução aos metodos discretos de apoio multicritério a decisão. Pioneira Thompson Learning, São Paulo.
  • 22. GOODALE JG. 1992. Improving performance appraisal. Business Quarterly, 57(2): 6570.
  • 23. GOODWIN P & WRIGHT G. 2000. Decision Analysis for Management Judgment, segunda edição. Wiley, New York.
  • 24. GUIMARÃES TA, NADER RM & RAMAGEM SP. 1998. Performance evaluation de pessoal: uma metodologia integrada ao planejamento e à avaliação organizacionais. Revista de Administração Pública, 32(6): 4361.
  • 25. HAMMOND JIS, KEENEY RL & RAFFA H. 2004. Decisões Inteligentes. Elsevier, Rio de Janeiro.
  • 26. HARKER P & VARGAS LG. 1987. The Theory of Ratio Scale Estimation: Saaty's Analytic Hierarchy Process. Management Science, 33(11): 13831403.
  • 27. HUNJAK T & JAKOVCEVIC D. 2005. AHP Based Model for Bank Performance Evaluation andR ating. In: ISAHP 2001, August 2-4, Berne.
  • 28]IENAGA CH. 1998. Competence-based management: seminario executivo. Dextron Consultoria Empresarial, São Paulo.
  • 29. ISLAM R & RASAD SM. 2006. Employee Performance Evaluation by the AHP: A Case Study. Asia Pacific Management Review, 11(3): 163176.
  • 30. KAPLAN R & NORTON D. 2005. The balanced scorecard Measures that drive performance. Harvard Business Review, 83(7/8): 172180.
  • 31. KATZ D & KAHN RL. 1970. Psicologia Social das Organizações. Atlas, São Paulo.
  • 32. KENNEDY PW & DRESSER G. 2005. Creating a Competency-Based Workplace. Benefits and Compensation Digest, 42(2): 1923.
  • 33. KIMURA H & SUEN AS. 2003. Ferramentas de Análise Gerencial Baseadas em Modelos de Decisão Multicritériais. RAE-eletrônica, 2(1): 118.
  • 34. LARRY B, CHARAN R & BURCK C.2002. Execution: The discipline of Getting Things Done.Crown Business, New York.
  • 35. LAWRENCE PR & LORSCH JW. 1973. As empresas e o ambiente: diferenciação e integração administrativas. Vozes, Petrópolis.
  • 36. LIMA MC. 2005. O método de pesquisa-ação nas organizações: do horizonte político à dimensão formal. Revista Eletrônica de Gestão Organizacional, 3(2): 141154.
  • 37. LIU C-H, HSU H-S, WANG S-T & LEE H-K. 2005. A Performance evaluation Model Based on AHP and DEA. Journal of the Chinese Institute of Industrial Engineers, 2151-7606, 22(3): 243251.
  • 38. LONGENECKER CO & FINK LS. 1999. Creating Effective Performance Appraisals. Industrial Management, 41(5): 1823.
  • 39. LOOTSMA FA. 1990. The French and the American School in Multi-critéria Decision Analysis. Recherche Operationelle/ Operations Research, 24(3): 263285.
  • 40. LUCENA MDS. 1977. Avaliação de desempenho: métodos e acompanhamento. McGraw-Hill do Brasil, São Paulo.
  • 41. MEADE LM & SARKIS J. 1999. Analyzing Organizational Project Alternatives for Agile Manufacturing Processes: An Analytical Network Approach. International Journal of Production Research,37(2): 241261.
  • 42. MEYER J, PAUNONEN SV, GELLATLY I, GOFFIN R & JACKSON D. 1989. Organizational commitment and job performance: it's the nature of the commitment that counts. Journal of Applied Psychology, 74(1): 152156.
  • 43. PHILLIPS LD. 1982. Requisite Decision Modelling. Journal of the Operational Research Society, 33: 303312.
  • 44. PHILLIPS LD. 1983. A Theory of Requisite Decision Models. Acta Psychologica, 56: 2948.
  • 45. PRAHALAD CK & HAMEL G. 1990. The core competence of the corporation. Harvard Business Review, 68(3): 7991.
  • 46. ROPÉ F & TANGUY L. 1997. Saberes e competências: o uso de tais noções na escola e na empresa. Papirus, Campinas.
  • 47. ROY B & BOUYSSOU D. 1993. Aide multicritèrie a la decision: methodes et cas. Economica, Paris.
  • 48. SAATY RW. 2002. Decision Making in Complex Environments: The Analytic Network Process(ANP) for Dependence and Feedback; A Manual for the ANP Software SuperDecisions Software. Creative Decisions Foundation, Pittsburgh.
  • 49. SAATY TL. 1991. Método de Analise Hierarquica. Makron Books do Brasil, Sao Paulo.
  • 50. SAATY TL. 1977. A Scaling Method for Priorities in Hierarchical Structures. Journal of Mathematical Psychology, 15(3): 234281.
  • 51. SAATY TL & VARGAS LG. 1984. The Legitimacy of Rank Reversal. Omega, 12(5): 513516.
  • 52. SHAOUT A & AL-SHAMMARI M. 1998. Fuzzy logic modeling for performance appraisal systems:a framework for empirical evaluation. Expert Systems with Applications, 14(3): 323328.
  • 53. SOMERICK NM. 1993. Strategies for Improving Employee Relations by Using Performance Appraisals More Effectively. Public Relations Quarterly, 38(3): 231241.
  • 54. STRAIGHT RL. 2000. Performance Metrics: avoiding the pitfalls. Public Administration Quarterly,23(4): 495516.
  • 55. SWIERCZ PM & EZZEDEEN SR. 2001. From Sorcery to Science: the AHP, a powerful New Tool for Executive Selection. Human Resource Planning, 24(3): 1526.
  • 56. TAYLOR S, BEECHLER S & NAPIER N. 1996. Toward an integrative model of strategic international human resource management. The Academy of Management Review, 21(4): 959985.
  • 57. TETT RP, GUTERMAN HA, BLEIER A & MURPHY PJ. 2000. Development and content validation of a hyperdimensional taxonomy of managerial competencies. Human Performance. Dayton, 13(3):205251.
  • 58. THIOLLENT M. 1997. Pesquisa-ações nas organizações. Atlas, São Paulo.
  • 59. THOMAZ HJR. 2006. Seleção de empregados em indústria de petróleo: uma análise pelo método AHP. Dissertação de Mestrado Profissionalizante apresentada ao Programa de Pós-Graduação em Administração das Faculdades Ibmec (Rio de Janeiro), como requisito parcial necessario para a obtenção do título de Mestre em Administração.
  • 60. VOSS C, TSIKRITSIS N & FROLICH M. 2002. Case research in operations management. International Journal of Operations & Production Management, 22(2): 195219.
  • *
    Corresponding authorIbmec/RJ, Av. Presidente Wilson, 118, Sala 1110, Centro, 20030-020 Rio de Janeiro, RJ, Brazil.E-mails:
  • Publication Dates

    • Publication in this collection
      08 Mar 2012
    • Date of issue
      Apr 2012

    History

    • Received
      11 Sept 2009
    • Accepted
      31 May 2011
    Sociedade Brasileira de Pesquisa Operacional Rua Mayrink Veiga, 32 - sala 601 - Centro, 20090-050 Rio de Janeiro RJ - Brasil, Tel.: +55 21 2263-0499, Fax: +55 21 2263-0501 - Rio de Janeiro - RJ - Brazil
    E-mail: sobrapo@sobrapo.org.br