Evaluating Supply Chain Management : A Methodology Based on a Theoretical Model

Despite the increasing interest in supply chain management (SCM) by researchers and practitioners, there is still a lack of academic literature concerning topics such as methodologies to guide and support SCM evaluation. Most developed methodologies have been provided by consulting companies and are restricted in their publication and use. This article presents a methodology for evaluating companies’ degree of adherence to a SCM conceptual model. The methodology is based on Cooper, Lambert and Pagh’s original contribution and involves analysis of eleven referential axes established from key business processes, horizontal structures, and initiatives & practices. We analyze the applicability of the proposed model based on findings from interviews with experts academics and practitioners as well as from case studies of three focal firms and their supply chains. In general terms, the methodology can be considered a diagnostic instrument that allows companies to evaluate their maturity regarding SCM practices. From this diagnosis, firms can identify and implement activities to improve degree of adherence to the reference model and achieve SCM benefits. The methodology aims to contribute to SCM theory development. It is an initial, but structured, reference for translating a theoretical approach into practical aspects.

Despite the increasing interest in SCM area, academics and practitioners still lack solid methodologies to guide and support SCM evaluation and implementation (Akkermans, Bogerd, & Doremalen, 2004;Croxton, Garcia-Dastugue, Lambert, & Rogers, 2001;Lambert, Cooper, & Pagh, 1998).Methodologies related to SCM implementation have been provided by consulting companies and are restricted in publication and use.A very important aspect of this development is that it should provide a framework for analysis, an efficient method for SCM field development and a clear explanation for practical applications, which can be considered of fundamental importance for researchers and practitioners (Chen & Paulraj, 2004;Wacker, 1998).
In this article, we propose a structured methodology to evaluate SCM practices, in order to explore this gap.This methodology must be based on objective criteria and must establish measurement scales that allows firms to analyze degree of adherence to an ideal SCM implementation.These criteria and scales are results of a deep literature review focused on identifying and selecting a SCM conceptual model as a reference.The proposed methodology was based on the conceptual model of Supply Chain Management proposed by Cooper, Lambert, and Pagh (1997).It involves eleven referential axes of analysis established from key business processes, SCM horizontal structures, initiatives and practices.
We expect to develop a tool that allows academics, consultants and decision-makers to assess firm maturity modeled on SCM practices, as an input to identify and implement actions to increase degree of firm adherence to the reference model and, consequently, provide improvements to the whole chain, making firms more competitive.
Article structure consists of five major sections besides this introduction.Section two summarizes the SCM theoretical framework.Section three discusses the selection of the SCM conceptual model used as reference to develop the methodology.Section four presents the research methodology.Section five presents the development of the methodology for evaluating companies' degree of adherence to a SCM conceptual model.Section six delivers main conclusions.

Supply Chain Management
Nowadays, most individual businesses no longer compete simply as autonomous organizations but rather as supply chains.A supply chain is referred to as a set of companies involved in the upstream and downstream flows of products, services, finances, and information from a source to a customer (Mentzer et al., 2001).In brief, it can be understood that the supply chain constitutes a network of business relationships.
From this point of view, as stated in section 1, the success of a single business primarily depends on its ability to integrate its network of business relationships.Thus, a source of competitiveness could be closely related to the supply chain in which the company is inserted.Christopher (1992) pointed out that currently the real competition is not company against company, but rather supply chain against supply chain.
In this context, Supply Chain Management (SCM) has emerged as a form of achieving adequate integration of a company's network of business relationships.Supply chain management extends the idea of integrating internal business functions, departments, and processes beyond the company's frontier to all companies in a supply chain (Cooper, Lambert, & Pagh, 1997;Fawcett, Magnan, & McCarter, 2008).Hammer (2002) suggest that companies that are able to work in close association with partners for project development and for the management of processes that involve the entire supply chain will succeed.
There are many definitions for supply chain management.Taking into account all the above mentioned aspects and specific objectives and purposes, The Global Supply Chain Forum definition was adopted for the development of this article: "Supply Chain Management is the integration of key business processes from end user through original suppliers that provides products, services and information that add value for customers and other stakeholders" (Lambert et al., 1998, p. 1).
The practicality and sectorial nature of SCM studies is remarkable in many publications.Authors used to analyze SCM implementation and/or make-or-buy decision as, for example, Ferreira and Padula's (2002) research into the beef supply chain, Silva and Fleury's (2000) work on the food industry, and Scavarda and Hamacher's (2001) investigation into the automotive industry.We intend to extend this discussion to reconcile both theoretical and practical perspectives, especially regarding SCM evaluation.
According to Lambert, Garcia-Dastugue, and Croxton (2005), only two of those conceptual models provide enough information to support research development in this important area: Cooper et al. (1997), and the Supply Chain Council (SCOR) models.
In this context, Cooper, Lambert and Pagh's SCM conceptual model (Cooper et al., 1997) was selected to support the evaluating methodology development proposed in this article, for many reasons.One is its high frequency of inclusion in existing research, much superior to the SCOR model.Another is the fact that it is defined broadly and abstractly enough to facilitate its potential study (Lewis, 1998).Additionally it is comprehensive, since it encompasses eight main business processes involving at least six functional areas.It has a more academic and didactic base since it clearly presents a conceptual structure and provides detailed information about the business processes which amplifies the possibilities for theory development.

Research Methodology
The approach used in this work combines three different stages and methodologies (Figure 1).First, we use a Discovery Oriented Approach (Kohli & Jaworski, 1990;Menon, Bharadwaj, Adidam, & Edison, 1999) to refine the preliminary model.This methodology involves discussing concepts in small groups of professionals, both academics and practitioners, in order to refine constructs.These professionals were selected based on their expertise in SCM, as shown on Table 1.All participants had, at least, five years of experience in SCM.After adjusting constructs and terminologies, which should match academic and managerial use, we selected three cases to pre-test the developed methodology.These firms, labeled A, B and C, were selected through purposive sampling.Firms A and B are Brazilian; Firm C is a multinational.All of them are leaders in their markets and has, at least, fifty years of operation.Firm profiles are presented in Table 2.

Firm Profile Characteristics
Firm A A 75-year old Brazilian manufacturer of machines for machining and injection molding of thermoplastics.Operates mainly in Brazil, but also in Latin America, North America, Europe and other countries.Exports around 15% of its production.It has an industrial park with eight plants.
No recognized expertise in Supply Chain Management, but operates in international markets by both exporting its products and importing components.
Firm B A 80-year old Brazilian manufacturer specialized in equipment for sugar and ethanol, pulp and paper, chemical and petrochemical, cement and mining, energy, biodiesel, food and brewery industries.It has ten factories located in four Brazilian regions.
No recognized expertise in Supply Chain Management; operates internationally with less intensity, exporting its products.
Firm C Multinational manufacturer and automaker of tractors and earth-moving machinery for construction of government projects, power plants, highways, railways, airports and reforestation projects, mining, agriculture and power generation markets.Exports about 75% of its production and is among the thirty largest exporters in Brazil, where it has operated for 55 years.
Company C is a successful case of Supply Chain Management.The Brazilian unit is an international benchmark in performance and operates well in international markets by exporting its products and importing components.
Finally, to get to the final product, we developed an iterative triangulation, employing systematic iterations between the literature review, case evidence and intuition based on researcher experience and judgment.
The methodology is better detailed during the development of each stage, in the next section.

Development of the Evaluating Methodology
In this section we detail all the stages necessary for constructing the methodology to evaluate companies' degrees of adherence to a SCM conceptual model.

Stage one: development of the preliminary methodology
Based on The Global Supply Chain Forum SCM definition, on the conceptual model of Supply Chain Management proposed by Cooper, Lambert and Pagh and on some basic SCM initiatives & practices, the methodology establishes eleven analysis referential axes.
The first nine analysis referential axes are related to key business processes and should identify whether the company manages and integrates them with key first-tier customers and key first-tier suppliers.Key business processes proposed by Cooper et al. (1997), Lambert, Cooper and Pagh (1998) and Croxton, Garcia-Dastugue, Lambert and Rogers (2001) are: .Customer Relationship Management; . Customer Service Management; . Demand Management; . Order Fulfillment; . Manufacturing Flow Management; . Supplier Relationship Management; . Product Development and Commercialization, and .Returns Management.
These processes must be coordinated by means of collaboration and relationship management along the supply chain, from initial suppliers to end consumers (Ballou, 2006).
In order to eliminate a possible source of confusion, the Returns Management process was separated into Returns Management from customers and Returns Management to suppliers.
The tenth referential axis is related to horizontal supply-chain structure and should identify whether the company monitors the management of key business processes beyond the first tier of key suppliers and the first tier of key customers.
The eleventh referential axis is related to SCM initiatives & practices and should identify whether the company uses or intends to use these initiatives & practices to support business processes management.
A defined number of requirements were associated with each referential axis.From the analysis of each requirement in each one of the referential axes it is possible to establish a company's degree of adherence to the SCM conceptual model.It is important to note that the core of the methodology is related to the integration of key business processes.

Stage two: developing model dimensions
In this stage, we develop and adjust the model's dimensions based on academy and industry perspectives.Requirements were associated with key business processes, horizontal supply chain structures, and SCM initiatives & practices.

Requirements associated to key business processes
Key business process definitions, objectives and strategic and operational sub-processes stated in literature (Bowersox & Closs 2001;Christopher, 2001;Croxton et al., 2001;Forslund, 2009;Lambert, 2004Lambert, , 2008;;Lambert, Stock, & Ellram, 1998;Sols, Nowick, & Verma, 2007) were detailed, analyzed, and translated into evaluating parameters or requirements using the jargon of the industrial environment.One hundred requirements were identified for key business processes.These requirements were submitted to a selected group constituted by an industry professional, an academic and three professors/professional consultants.After many discussions and meetings, we defined which requirements should be considered in the methodology.The final requirements are presented in Tables 3 to 13.

Requirements of the Referential Axis Related to the Customer Relationship Management Business Process
Process: Customer Relationship Management

Requirements:
. Company differentiates customers by identifying key customers .Company has a cross-functional team for customer relationship management .Company provides the framework for managing relationships with customers .Develops and implements customized product/service agreements for key customers .Develops and implements product/service agreements for customer segments .Develops process improvement programs with customers .Develops efforts to reduce demand variability with customers .Develops programs with customers to eliminate non value-added activities .Identifies opportunities with customers .Develops guidelines for sharing process improvement benefits with customers .There are procedures to periodically evaluate customers, based on products bought, sales and positions in their market segments .Designs performance reports to measure the firm's financial impact on customers .Designs performance reports to measure customer's financial impact on the company Table 4

Requirements of the Referential Axis Related to the Customer Service Management Business Process
Process: Customer Service Management

Requirements:
. Company has a cross-functional team for customer service management .Develops customer service strategies .There is a communication channel to provide customers with information about PSA .There is an action plan to evaluate alternatives for managing events and their effects on customers and on the company's internal operations .The team determines a set of actions by working jointly with specialists in each of the functions affected by the event to evaluate the situation and define solutions .The team coordinates the implementation of selected alternatives .The event is recorded in a database that can be used for future reference .The evolution of the event is monitored in order to know to what extent the response has been implemented .The team informs customers about how the issue is being resolved

Requirements:
. Company has a cross-functional team for demand management .There are guidelines to collect data/information for demand forecast .The team determines which forecasting approaches to use .The forecasts are communicated to the other process teams that are affected by them, including customer service management, order fulfillment, manufacturing flow, and product development and commercialization .There are synchronization procedures to match the demand forecast with the company's production, sourcing and distribution capabilities. .There are contingency plans in the event of internal or external events that disrupt the balance of supply and demand .The team reviews and adjusts forecasts periodically .The team works with the manufacturing flow team to gain flexibility .The team works with the customer relationship management team to reduce demand variability .The team periodically evaluates differences between forecasts and actual demand .The team periodically evaluates the level of production capacity utilization Table 6 Requirements of the Referential Axis Related to the Order Fulfillment Business Process Process: Order fulfillment

Requirements
. Company has a cross-functional team for order fulfillment management .The team defines requirements for order fulfillment .The team defines the specific steps from order entry to product delivery .The team verifies customers' credit .The inventory is checked to verify if it is possible to fulfill a customer's order .The team communicates with the Customer Service Management team when a customer's order is delivered .The team communicates with the customer relationship management process team to make sure that all customer expectations are met. .Order cycle time is measured .The team monitors quantity of complete orders delivered on time and compares this to the order fulfillment policy .The team provides information about the order to the Customer Service Management, Manufacturing Flow Management and Demand Management teams Table 7

Requirements of the Referential Axis Related to the Manufacturing Flow Management Business Process
Process: Manufacturing Flow Management

Requirements:
. Company has a cross-functional team for manufacturing flow management .The team develops and implements a master production schedule

Requirements of the Referential Axis Related to the Returns Management Business Process (To Suppliers)
Process: Returns Management (to suppliers)

Requirements:
. Company has a cross-functional team for the returns (to suppliers) management process .There are defined procedures for the returns management process .There are defined plans for transporting and holding returned products until they reach their final disposition .There are rules for supplier credit/debt .There is a program for returnable packaging .There is an analysis plan to identify causes of returns .There are procedures for evaluating financial impact of returns .The team informs the supplier relationship management process team about returns .There are procedures for eliminating causes of returns

Requirements associated with horizontal supply chain structures
Supply chain management currently occurs to a very limited degree.This means that the most likely place for SCM to take place is between the company and its first-tier suppliers and customers (Ballou, 2006).
Nevertheless, for horizontal supply chain structures, which identify whether the company monitors the management of key business processes beyond first-tier key suppliers and first-tier key customers, four requirements were defined:

Requirements of the Referential Axis Related to Horizontal Supply Chain Structures
Referential axis related to horizontal supply chain structures: Requirement: .Company monitors key business process management with a second tier of key customers .Company monitors key business process management beyond the second tier of key customers .Company monitors key business process management with a second tier of key suppliers .Company monitors key business process management beyond the second tier of key suppliers

Requirements associated with SCM initiative & practices
For initiatives & practices, which identify if the company uses or intends to use SCM initiatives & practices to support business process management, eight requirements were defined.

Categories and measurement scale
A set of categories is associated with each requirement of each referential axis.So, each requirement is classified into five categories, and each category reflects the company situation related to that specific requirement.
The categories combine characteristics of amplitude and depth.Amplitude is related to the quantity of items to which the requirement is applied and is expressed in two levels: (a) for the majority of the items, and (b) not for the majority of the items.Depth is related to the way the requirement is applied: in a documented manner (formally) and in a non-documented manner (informally).
Considering these situations, the following categories were established: . Category 5 -the company satisfies the requirement for the majority of the items related to it and documents it formally.
. Note: The items related to the requirement refers, for example, to customers, orders, events, teams, suppliers, functional areas, departments, and so on.
. Category 4 -the company does not satisfy the requirement for the majority of the items related to it but documents it formally; . Category 3 -the company satisfies the requirement for the majority of the items related to it but does not document it formally; . Category 2 -the company does not satisfy the requirement for the majority of the items related to it and does not document it formally; . Category 1 -the company does not satisfy the requirement.
As a result, category 5 reflects the best situation concerning the parameters toward the high degree of adherence by the company to the SCM conceptual model.To the contrary, Category 1 reflects the worst situation.Considering that it is not possible to measure the effort necessary to take one requirement from a defined category to a higher one, we adopted an ordinal measurement scale.This provides information about the ordination of categories, not about the magnitude of the differences among them (Rea & Parker, 2000).

Analysis of the results
Normally companies apply many of the requirements of business processes, but this does not mean that they have successful SCM.What really matters is the integration and management of all business processes, and to do this, companies must apply all the requirements of all processes, for the majority of the items related to each requirement, and in a documented manner (i.e.formally).This is called the highest ordination and this is the situation that can lead companies to the highest degree of adherence to the SCM conceptual model.So, the degree of company adherence is obtained in dependence with the frequency of requirements occurring in the highest ordination, taking into account all referential axes.Table 14 helps organize the data and Table 12 provides the degree of adherence.
The percentage value obtained in ordination 5, in Table14, represents adherence (A).From this value, on Table 15, the degree of company adherence to the SCM conceptual model is obtained.From the total of 112 requirements to be evaluated, those related to key business processes and horizontal structures of a supply chain are mandatory.From those related to SCM initiatives & practices, only one, Supplier and Customer base Reduction and Consolidation, is mandatory (companies can manage their supply chains without considering the others, even though they are helpful).In this way, 105 requirements are mandatory, which means that ideal adherence A is situated in a range from 94 to 100%.High and medium adherence was established at a range of 10 percentage points from 84 to 94%.Low adherence is equal to or below 74%.
Frequency of requirements occurring in ordinations 1 to 4 must be used as a reference to identify the status of the company related to SCM.From them, a company should analyze requirements involved in these ordinations, verify which are the more distant of the 5 ordinations and which are the most critical.A company must establish priorities and action plans toward a higher degree of adherence.Even though the adopted ordinal scale is not aimed at measuring the effort to go from one level to a higher one, ordinations 1 to 4 help a company to have, at least, an idea of the overall situation.
It should be noted that Tables 3 to 15, presented above, are the final ones since they already integrate the knowledge and suggestions obtained from application.

Organization and planning of applying the methodology
The requirements established for the referential analysis axis form the questions that are put together in a questionnaire built on a structure aiming to facilitate its application and motivate respondents.
Considering the complexity and the high number of questions, the data collection instrument should be applied by personal interviews.In this manner the researcher can get more detailed information, explain the questions, provide detailed instructions, and assure response reliability by checking evidence.On the other hand, there are disadvantages when using this method; for instance, higher costs, more stress and less privacy (Rea & Parker, 2000).In these applications, disadvantages are easily compensated for by the advantages, mainly taking into account that SCM is a new field and needs practical and theoretical complementation.
The methodology application should begin with a detailed planning of the interviews, previously selecting the people that will be interviewed in a company and arranging these interviews.

Identifying the focal company
The methodology considers that a supply chain is a group of basic chains, and each basic chain involves the company that will be evaluated and its first-tier customers and suppliers.In way, each company must manage and integrate business processes within key members of their basic chain and then the whole chain will be managed.The lead company of the supply chain is responsible for verifying or monitoring whether the other key members, both suppliers and customers, are managing their basic chains.Therefore the evaluating methodology should be applied at the supply-chain lead company.

Development of adjusted methodology
After its development, the preliminary methodology was pretested to evaluate the defined requirements, categories, measurement scale and matters related to the interviewees' understanding of the questions, clarity of construction and readability, comprehensiveness and focus of requirements and related categories and acceptability.Based upon the results of the pretest, some items were eliminated or reworded, and others were added.
For this purpose four academic and three industry experts were interviewed.

Development of evaluating methodology
As initially stated, an illustration application was conducted in three relevant companies to accomplish Forza's (2002) and Lewis' (1998) recommendations, for refining the methodology and for demonstrating its applicability.
For this purpose, an intentional sampling was established.Intentional sampling is a nonprobabilistic sampling in which the researcher utilizes professional experience to select, for instance, the companies that will participate in a defined research (Rea & Parker, 2000) rather than selecting them at random.The companies included one with recognized competency in SCM, and two companies without recognized competency in the area.However, the three companies are leaders in their respective supply chains.
After the illustration application the knowledge gained was analyzed and added to the adjusted methodology, resulting in the final methodology for evaluating company degree of adherence to a SCM or SCM -EAD Methodology conceptual model.

Conclusions
Although SCM today is undoubtedly a contemporary and important area in the field of operations management, the literature still reveals a paucity of academic studies involving key topics, such as generic methodologies to guide and support SCM implementation and evaluation.
The scientific development of a structured supply chain management discipline requires that advances must be made in the development of measurement tools as well as in theoretical models to improve supply chain management understanding.
This paper makes several contributions to the knowledge of supply chain management evaluation.In terms, the methodology proposed in this article can be considered to be a diagnostic instrument that permits companies to evaluate their status concerning SCM aspects.Based on this diagnosis, companies can identify and implement activities aimed at increasing their adherence to the reference model and augmenting the benefits gained through SCM.
The methodology provides a model to analyze SCM, contributes to the development of the subject and provides explanations for practical applications.Additionally, it can be used by both academics and practitioners to develop complementary research in this area.
Derived from a major research project, the methodology contributes to the theoretical development on SCM and research development on applications of structured models for implementing and supporting effective SCM.
The methodology is an initial reference for the SCM evaluation process, and should be improved as SCM theory evolves.Since this theory is still under construction, many issues need to be clarified and defined, which could then be extended to application in firms.The proposed methodology is limited in that it doesn't contemplate the level of importance that each business process has to a firm.Another limitation rises from the fact that the types of manufacturing processesi.e.mass production or customizedare not included in the proposal.
However, we expect that such limitations will be overcome in that the theory on the subject will develop and extend the practical application of this methodology.Future research, in addition to exploring the limitations outlined above, may seek to validate the proposed categories and scales, via more comprehensive and extensive studies of different types of companies and supply chains.

Table 8 Requirements of the Referential Axis Related to the Supplier Relationship Management Business Process
Develops and implements product/service agreements for supplier segments .Develops process improvement programs with suppliers .There are procedures to quantify process improvement benefits .Identifies opportunities with key suppliers .Develops guidelines for sharing process improvement benefits with suppliers .Designs performance reports to measure the costs/profitability of individual supply orders .There are procedures for supplier evaluation and development .There are procedures to periodically evaluate suppliers, based on products bought, relationships and positions in their market segments Process: Supplier Relationship ManagementRequirements:. Company differentiates suppliers by identifying key suppliers .Company has a cross-functional team for supplier relationship management .Company provides the framework for managing relationships with suppliers .Develops and implements customized product/service agreements for key suppliers .