Destruction of measurement scale through exploratory factor analysis in production and operations research

Abstract

This paper aims to assess the use of Exploratory Factor Analysis by Production and Operations researchers, discussing the adequacy of its application. We analyzed 97 papers published between 2010 and 2015 in the Production and Operations area -- of which 61 and 36 were published in international and Brazilian journals, respectively. These papers contain 140 different applications of Factor Analysis. The research shows that confirmatory techniques are prevalent in international papers, as well as exploratory techniques to evaluate the problem of common method bias. Conversely, the papers in Brazilian journals typically use the exploratory technique in more traditional ways, such as to confirm the unidimensionality of the construct, or still to generate scores for use in other statistical techniques. Despite the textbooks for the AFE teaching focus exclusively on the use of AFE in the exploratory mode (to identify the number and meaning of the common factors), this use has been less frequent in published articles, both national and international. Moreover, the research shows that the inappropriate use of exploratory (rather than confirmatory) factor analysis in four Brazilian papers resulted in the “destruction of theory”. These findings suggest that national research have been using exploratory factor analysis in a questionable way; in this sense we propose scholars discuss this topic in order to disseminate the good practices.

Keywords:
Multivariate analysis; Statistical methods and models; Exploratory factor analysis; Confirmatory factor analysis; Measurement scales in productions and operations research

Resumo

O presente artigo teve como objetivo avaliar o uso da análise fatorial exploratória (AFE) nas pesquisas da área de Produção e Operações, discutindo a adequação de sua utilização. Foram analisados 97 artigos (61 internacionais e 36 nacionais), totalizando 140 aplicações da análise fatorial (AF), no período de 2010 a 2015. Verificou-se que, nos artigos internacionais, predomina o uso de técnicas confirmatórias e a aplicação da AFE para se avaliar o common method bias, ao passo que, nos artigos nacionais, a técnica foi utilizada com funções mais tradicionais, como, por exemplo, a avaliação da unidimensionalidade ou ainda a geração escores, para uso em outras técnicas. Apesar de os livros didáticos para o ensino de AFE focarem, exclusivamente, no uso da AFE de modo exploratório (identificar a quantidade e o significado dos fatores comuns), este uso tem sido o menos frequente nos artigos publicados, tanto nacionais como internacionais. Apurou-se, ainda, que, em quatro artigos nacionais, houve “destruição de teoria” ao se usar AFE, quando deveria ter sido utilizada a AF confirmatória. Estes resultados indicam que as pesquisas nacionais têm feito uso questionável da técnica, o que sugere a necessidade de discussão desse tema entre os acadêmicos, de forma a se difundirem as boas práticas.

Palavras-chave:
Análise multivariada de dados; Métodos e modelos estatísticos; Análise fatorial exploratória; Análise fatorial confirmatória; Escalas de mensuração em produção e operações

1 Introduction

Books that are written to teach exploratory factor analysis (EFA), for instance: Aranha & Zambaldi (2008), Fávero et al. (2009), Hair et al. (2005, 2010), as well as Malhotra (2012) – typically describe two main applications for EFA: (i) data reduction, which consists of the procedures that group a number of variables into a smaller number of factors, which explain as much as possible the variance of the original data; in general, the use of principal components analysis is recommended in this case, as the method of extraction; (ii) identification of factors that are not measured directly, and which produce correlations that are observed in the operational variables (also known as indicators or items); to that purpose, it is recommended that common factor extraction methods be used, such as principal axis factoring (PAF), maximum likelihood (ML), image factoring, alpha factoring, generalized least squares minimum, or unweighted least squares.

Both aforementioned uses occur in an exploratory research context, which Osborne (2014) explicitly mentions in a number of occasions:

Let us repeat again our mantra for this book: EFA is exploratory... (location 990 of the e-book).

Keep the ‘E’ in EFA! (location 5154 of the e-book).

In spite of its widespread application in business administration research, there appears to be some confusion (which we will be addressed in this article) with regards to the objectives of the use of exploratory or confirmatory techniques (hereinafter, EFA and CFA).

The exploratory use (EFA) should be deployed when the researcher holds little or no knowledge with regards to the latent structure that comprises the set of research indicators − in other words, when there is no empirical research supporting it, or when the theory that supports the phenomenon is incipient, so that there is little prior knowledge regarding the constructs' dimensionality or its composition. In this case, the exploratory use allows the researcher to understand the latent structure, and represents an advancement in the scientific knowledge for the field (Jöreskog, 2007; Pasquali, 2012; Pett et al., 2003).

In contrast, the confirmatory use (CFA) is prescribed when the theory that describes the given phenomenon is consolidated, and the ensuing studies replicate the scale (which is the subject of this paper). In this manner, the researcher already holds abundant knowledge about the phenomenon and its inter-relations, so that the factor analytical techniques are used to empirically test the model using the given sample data. In this context, the structural factors are known from the literature a priori, as well as the number of factors to be measured, and their meaning (Jöreskog, 2007; Pasquali, 2012; Pett et al., 2003).

In spite of this issue regarding the use of exploratory and confirmatory techniques, in 2014 a discussion panel was carried out in the EnANPAD congress. During that meeting, scholars that are active in the area of quantitative methods questioned the adequacy of the use of EFA in business administration research, particularly with respect to the application of exploratory techniques for measurement scales that had already been published in the scientific literature (Bido, 2014). The expression “destruction of theory” emerged from that perspective, corresponding to the situation where EFA is used, in lieu of the more appropriate CFA technique.

A survey was conducted by Mantovani & Bido (2015) in peer-reviewed top-tier (QUALIS A2) publications in business administration to investigate this practice, and found an inadequate use of EFA in 59% of the FA applications.

Such findings bring to light the limitations imposed by the use of exploratory techniques in scientific research, leading to significant outcomes. Specifically: i) loss of comparability of results with previous research, which is fundamental for furthering the research contributions, so that they can be revealed and discussed by academia; and ii) a consequent disregard for the existing theory, so much so that the conceptual meaning of the constructs is lost, in some cases. This justifies the extension of this inquiry to a broader context, that is pertinent to specific areas of business administration − such as Production and Operations.

In view of the above, the objective of this work is to evaluate the application of the EFA in Production and Operations research, through a broad review of the publications in Brazilian and international journals. The aim is to identify the manner with which the technique was deployed, and identify the cases in which the theory was destroyed (that is, when the dimensionality of scale from the original theory or previous research was not preserved).

The following section presents the literature review related to the potential applications of EFA, in order to discuss the proper decisions and the corresponding methodological procedures, followed by a description of the results found, general discussion and research conclusions.

2 Applications of exploratory factor analysis

Authors like Conway & Huffcutt (2003) and Fabrigar et al. (1999) lay out the good practices of the EFA applications. Yet, before we probe into the adequate use of EFA - that is, before we discuss how appropriate the adjustment indicators are -, it is necessary to assess whether the exploratory techniques are indeed appropriate in light of the proposed objectives.

In this context, the work of Hurley et al. (1997) presents a debate among seven scholars related to this question, leading to the conclusion that the choice between EFA or CFA depends on the research objectives. In addition, these scholars point out the importance of the research decisions which support this decision, stating that the CFA should not be applied in an exploratory way. In conclusion, they posit that a support theory is essential to justify the application of the exploratory or confirmatory techniques (Hurley et al., 1997).

2.1 EFA used in a purely exploratory manner

The exploratory approach must be “data oriented”; that is, its use is recommended when there is no previous knowledge whatsoever about the number of factors to be extracted. Hence, EFA is recommended when there is little or no restriction on how indicators come together in the formation of factors. In other words, the exploratory approach begins with the data, in search of an optimal factorial solution that best represents the structure of interrelationships between the variables (Fabrigar et al., 1999). This implies that there is no supportive theory guiding the analysis, nor any previous empirical research that might suggest any expected outcomes (Pasquali, 2012).

In the same vein, Worthington & Whittaker (2006) postulate that EFA is useful in research geared to the development of measurement scales, since the dimensionality of the constructs is based solely on the responses of the sample cases. Accordingly, it is possible to identify items that do not contribute for the measurement of the expected construct; or conversely, the items load in more than one construct (i.e., cross-loadings). Since the scale development is a dynamic process, it involves constant review and refinement, and consequently EFA is the recommended technique for such applications.

EFA applications in truly exploratory contexts are commonly found in the Quantitative Methods textbooks for undergraduate and graduate-level business administration courses, for example: Fávero et al. (2009), Hair et al. (2005) and Malhotra (2012).

Although it is beyond the scope of this paper to evaluate its accuracy, it is worth noting that caution is suggested regarding its proper use. Kaiser (1970) draws attention to the frequent and indiscriminate use of principal components analysis, as well as the use of the Eigenvalue criterion (greater than one), and lastly suggests caution when using the Varimax orthogonal rotation procedure (known in the literature as “Little Jiffy”).

Almost 50 years ago, Kaiser pointed out that the exploratory objective of research implies the existence of a theoretical knowledge gap. In this context, the author postulates that researchers should use common factor extraction algorithms, as well as the oblique rotation, which allows the extraction of correlated factors − a more realistic solution than the uncorrelated factor assumption, i.e. the orthogonal rotation.

Therefore, the use of “Little Jiffy” (principal components, eigenvalue greater than 1 and Varimax rotation) were coded as 6, 7 and 8 in this work, respectively. These are considered questionable uses for the purposes of this article (see Chart 1).

Chart 1
Categorization of the factor analysis applications.

Types of use of factor analysis	Adequacy
1 = CFA, SEM, PLS-PM – Did not use EFA	Not evaluated
2 = EFA for Harman’s test (extract one factor to test CMV)	Legitimate
3 = EFA to test unidimensionality (1st factor with eigenvalue > 1 and 2nd factor with eigenvalue < 1)	Legitimate
4 = Deductive EFA (Hinkin) and confirm theory (knows beforehand which latent variables, runs EFA and finds the latent variables) or coincidence	Questionable
5 = Deductive EFA (Hinkin) and destroyed theory (knows beforehand which latent variables, runs EFA and finds other latent variables)	Questionable
6 = Exploratory EFA: common factor (PAF, ML...) + oblique rotation (does not know beforehand which latent variables)	Legitimate
7 = Exploratory EFA: principal components + Varimax (Little Jiffy) (does not know beforehand which and how many latent variables)	Questionable
8 = Exploratory EFA: other combinations of extraction and rotation (does not know beforehand which and how many latent variables)	Legitimate
9 = EFA with instrumental use (generate orthogonal scores to use them in regression, evaluate multicollinearity etc.)	Legitimate

• Source: Developed by the authors.

It is worth mentioning that various statistical packages often use Little Jiffy as the default (pre-selected) option for EFA, which may explain the continuous, massive use of this procedure that is considered inadequate.

Although Kaiser (1970) and Pett et al. (2003) point out that the common factors analysis and principal components (PCA) are factor analysis algorithms, some authors do not consider PCA a type of factor analysis per se (Costello & Osborne, 2005; Fabrigar et al., 1999). This perception reinforces the need for careful examination regarding the choice of the extraction method.

2.2 Uses of EFA that are not found in academic books

In addition to the described exploratory applications, EFA can be used for different purposes, which may be legitimate or questionable, for instance: instrumental use to obtain orthogonal factors; evaluation of multicollinearity; evaluation of the unidimensionality of the constructs; Hinkin's deductive method (1995, 1998) for the development of scales; Harman test for common method bias evaluation; and lastly, the improper use of EFA in lieu of CFA. These are discussed below.

2.2.1 Instrumental use

The instrumental use of EFA leads to the composition of the orthogonal factor scores, which can be used as input to other statistical analysis - such as regression analysis, noting that this technique could be impaired by the occurrence of multicollinearity; in this case, we would be interested in addressing the multicollinearity issue. Here, the use of EFA is considered legitimate (code 9 in Chart 1 of the Methodological Procedures section). This type of application is cited and encouraged in the multivariate analysis literature (Fávero et al., 2009; Hair et al., 2010).

To eliminate multicollinearity, Cohen et al. (2003, p. 428-429) prescribe the use of principal components regression, which is equivalent to replacing the original independent variables with their unrotated, orthogonal principal components. Notwithstanding the fact that these procedures addresses the effects of multicollinearity, the authors recognize that the interpretation of the regression coefficients could be impaired.

2.2.2 Evaluation of unidimensionality

Conway & Huffcutt (2003) discuss the use of EFA for the unidimensionality test of a scale or construct. In a way, this can be viewed as the confirmatory use of an exploratory technique.

In this context, the researcher knows beforehand that there is one single factor, and the EFA is applied to verify if one single factor is indeed extracted. One commonly used criterion for this test is the first unrotated extracted factor with an eigenvalue much greater than one, or extracted variance greater than 50%, as well as the second extracted factor with an eigenvalue lower than one. It should be noted that this procedure (code 3 in Chart 1) is legitimate only for one-dimensional scales. In other words, it is incorrect to apply the EFA for each single factor on a multidimensional scale, since this procedure does not test for discriminant validity; the use of CFA is recommended in this case.

This technique is used for the estimation of structural equation models with partial least squares estimation, as shown in Figure 1.

Figure 1
Unidimensionality in the software. Source: Figure (a) Sanchez (2013, p. 56); Figure (b) XLSTAT (2017a). Note: Description of unidimensionality tests in XLSTAT (2017b).

2.2.3 Scale development using Hinkin’s deductive method

Hinkin (1998, p. 106) describes the development of scales using the deductive and the inductive approaches:

Scale development in the deductive manner derives its name from the fact that the theoretical foundation provides enough information to generate the initial set of items.

The inductive approach may be appropriate when the conceptual basis for a construct does not necessarily result in easily identifiable dimensions, through which the items are generated. Researchers often develop scales inductively by interviewing a sample of respondents to provide descriptions of their feelings about the organization, or to describe some behavioral aspect (Hinkin, 1998, p. 107).

Hinkin (1995, 1998) proposes that the deductive method be used to generate items for the scale development. An assumption of this method is that the researcher already knows the structure of interrelationships between variables, as well as the theoretical meaning of the constructs – which facilitates the development of the indicators. This being the case, the researcher goes to the field to collect data, and applies the EFA to verify if the initial latent structure is confirmed. Otherwise, new items are generated until the previously known theoretical structure is confirmed.

For the purposes of this work, the use of the deductive method is considered legitimate only when the researcher refers to it in an explicit manner. In cases where EFA is applied in known scales (but with no mention of the deductive method), the assumption is that the solution which coincides with the original scale was found by chance. In other words, this type of application is considered questionable (code 4 on Chart 1).

2.2.4 Use of EFA, when CFA is recommended (questionable use)

The use of EFA may lead to the “destruction of theory”, which corresponds to the situation where the researcher applies the exploratory technique to a known scale. In other words, the researcher knows the constructs and the number of factors, but he or she obtains different factors than those found in the literature and previous research; however, the results are considered definitive (code 5 in Chart 1).

This use of EFA bears some similarity with Hinkin's deductive approach, but they are in fact distinct. In Hinkin's deductive approach, the researcher continues to conduct research, generating new items and collecting new samples, until the items that adequately represent the dimensions (defined a priori) are obtained. When destruction of theory occurs, the dimensions obtained are different from those expected, yet the researcher still publishes the results (Bido, 2014; Mantovani & Bido, 2015).

2.2.5 Common method bias – Harman’s test

When estimating structural models, it is common to use survey data. If the data from the exogenous and endogenous variables is obtained from the same source (same respondent, same format of responses, same form of collection, and even the same moment), some form of common method bias may be present. This is an issue that leads to the underestimation or overestimation of the structural coefficients, or of the relations between latent variables. This bias occurs mainly because the method itself is the common cause between the dependent and independent variables.

Podsakoff et al. (2012) suggest various procedures to prevent this bias, while Chin et al. (2013) propose a method to estimate and control it.

Even though the Harman test is criticized by authors like Guide & Ketokivi (2015) − because it does not test or control the bias of the method -, it has been traditionally used in business administration research.

In Harman’s single factor test, both the structural model (that deals with the relations between latent variables) and the measurement model (which contains the relationships between indicators and their latent variables) are disregarded. Subsequently, the EFA technique is applied to all items, and an unrotated principal components method is produced. Harman’s test considers that there is bias when the solution results in one single extracted factor, or when a single factor extracts most of the variance from the set of variables tested (Podsakoff et al., 2003, 2012). This is considered a legitimate use of EFA (code 2 in Chart 1).

2.2.6 Regarding the uses and “new” uses of EFA

An important contribution of this article is the acknowledgement of the legitimacy of specific EFA uses, beyond its original intent.

It could be argued these “new” uses are more adequate for books that explore confirmatory factor analysis and structural equation modeling in depth. Nonetheless, EFA textbooks ought to begin explaining these uses, while still recognizing the instances where EFA is adequate, as discussed by Pett et al. (2003, p. 3).

Similarly, it is important to recognize the inadequate uses of EFA, which lead to the destruction of theory. The dimensionality of scales developed in other contexts must be taken into account due to its relevant effects. This issue must be diagnosed and corrected, as it propagates the misuse of exploratory techniques in academia, in addition to bringing difficulties in the evolution of the theories in the business administration area.

3 Methodological procedures

In light of these uses of the analytical techniques, the present study was developed in three stages: construction of an analysis protocol; survey of the articles published in national and international peer-reviewed journals; and presentation and discussion of results.

3.1 Analysis protocol

The analysis protocol was developed according to the theoretical framework, but was adjusted as the articles were evaluated. The first version is comprised of three general categories, namely: EFA used in an exploratory way; EFA used in confirmatory mode and CFA used in confirmatory or exploratory mode (which were not analyzed further). Subsequently, the EFA used in exploratory mode was divided into three additional application categories (which correspond to codes 6, 7 and 8 of Chart 1, respectively). When EFA was used in a confirmatory manner, it was coded 2, 3, 4, 5 and 9 − adding up to a total nine types of applications.

The papers that use only CFA or structural equation modeling (SEM) with covariance-based estimation or variances were given code 1. These were not further analyzed, due to the fact that they were beyond the scope of the research. For all other cases, the use of EFA was identified and coded according to the criteria that is shown on Chart 1.

It is worth noting that one same article might have performed multiple applications of EFA. In these situations, for each individual use of the exploratory technique, the use was evaluated separately. In case the use is common to all applications, the paper received one single code.

Chart 1 presents three applications that are categorized as questionable, followed by the appropriate justifications:

• Application #4: In specific cases, the authors make clear that the conceptual framework of the constructs is already known, and that EFA preceded CFA. The authors remove indicators that have low factor loadings, but maintain the measurement model as defined in the theoretical framework. In other cases, however, this process seems to occur by chance (i.e., it is a matter of luck);

• Application #5, called “theory destruction”, is the simplest and most complete description of these cases, in which there is a measurement model known in advance. However, after being modified subsequent by the exploratory analysis, it loses its comparability with the previous studies: clearly, CFA or SEM should have been used, rather than EFA;

• Application 8: In general, the recommendation is to use common factor extraction and oblique rotation methods (Conway & Huffcutt, 2003; Fabrigar et al., 1999).

In order to ensure the reliability of the analysis, one of the authors analyzed the national articles, the second author analyzed the international articles, and the third author verified the analysis made by the former authors. This was followed by a discussion of the differences, which in turn allowed the authors to reach a consensus regarding the categorization of the application of FA in the articles.

3.2 Survey of the articles published in Production and Operations journals

The criteria used to select the journals to be evaluated took into account the following: (i) specific publications of the area of Production and Operations; (ii) publication indexed and evaluated by Qualis, or JCR (Web of Science) impact factor. Accordingly, we selected articles published in four national journals and six international journals (Table 1).

Table 1
Papers analyzed by year and source.

Publication	Qualis or JCR	2015	2014	2013	2012	2011	2010	Total
(a) National		4	6	5	8	6	7	36
Revista Produção Online	2014 = B4 2012 = B2	1	1	1	2	1	-	6
Gestão & Produção	2014 = B1 2012 = A2	2	3	2	4	4	2	17
Journal of Operations and Supply Chain Management	2014 = B3 2012 = B3	-	-	-	2	1	4	7
GEPROS	Adm.=B3 Eng. III=B4	1	2	2	-	-	1	6
(b) International		51	10	-	-	-	-	61
Production and Operations Management	Eng. III=A2 JCR = 1.439	10	-	-	-	-	-	10
Journal of Operations Management (JOM)	JCR = 3.818	10	-	-	-	-	-	10
International Journal of Operations & Production Management	Eng. III=A2 Adm.=A1 JCR = 1.736	10	-	-	-	-	-	10
Journal of Supply Chain Management	JCR = 3.857	7	3	-	-	-	-	10
Supply Chain Management: An International Journal	Eng. III=A1 JCR = 3.500	8	3	-	-	-	-	11
Journal of Business Logistics	JCR = 1.833	6	4	-	-	-	-	10

• Notes: JCR = Impact factor of 2014. The list of national papers can be found in ^{Appendix A}; the international papers can be found in ^{Appendix B}. Source: data compiled by the authors.

The search ranged from papers published between 2010 and 2015. Initially, search engines were used to search the journal website, using as search criterion the following expression: “factor analysis” OR “análise fatorial” OR “principal components” OR “componentes principais” OR “KMO” OR “scree plot” OR “eigenvalue” OR “autovalor”.

As shown in Table 1, factor analysis is most frequent in international journals, and the average number of papers per publication in 2015 was 8. As a result, a decision was made to analyze approximately 10 recent articles from each publication, which was considered to be a sufficient sample to support the analysis of the application of the FA technique.

As for the national publications, the average is 1.5 articles per journal per year for 2015. It is worth noting that the quantity identified by the search engines was much smaller. In light of that finding, the authors supplemented this process by way of a manual search, in order to verify all articles published during that specific time period.

4 Results and discussion

Initially, a comparison is made between EFA in national and international applications. The significance of the chi-square test (p < 0.001) confirms the evidence presented in Figure 2: there is a significant difference between the applications of factor analysis between national and international articles. International papers have more frequent use of confirmatory techniques and Harman's test, in comparison with national papers in the area of Production and Operations.

Figure 2
Use of factor analysis in national and international papers. Note: Type of application: 1 = CFA, SEM, PLS-PM; 2 = EFA for Harman's test (extract one factor to test common method variable); 3 = EFA to test unidimensionality (1^st factor with eigenvalue > 1 e 2^nd factor with eigenvalue < 1); 4 = Deductive EFA (Hinkin) and confirm theory (know a priori which are the latent variables, run EFA and find latent variables); 5 = Deductive EFA (Hinkin) and destroy theory (know a priori which are the latent variables, run EFA and finds new latent variables); 6 = Exploratory EFA: common factor (PAF, ML...) + oblique rotation (does not know beforehand how many and which latent variables); 7 = Exploratory EFA: principal components + Varimax (Little Jiff) (does not know beforehand how many and which latent variables); 8 = Exploratory EFA: other combinations of extraction and rotation (does not know beforehand how many and which latent variables); 9 = EFA with instrumental use (to generate scores for subsequent use in regression, to reduce multicollinearity, etc.). Source: Survey data.

International papers have on average 1.6 FA applications per article, and the use of CFA along with EFA for the Harman test is prevalent. More specifically, 51% of articles use more than one FA application, namely: six articles with three different applications, 25 articles with two different applications, and the other (30) use one single application.

For national papers, the average is 1.2 application per article. Thus, there are five articles (12%) with two different applications of FA, and the others (32) only one. For the national papers, the is more frequent use of the EFA for 1) evaluation of the unidimensionality of the constructs, 2) identification of factors by Principal Components method of extraction and orthogonal rotation (Little Jiffy), and 3) instrumental use. These may be considered more traditional applications of the technique, and are presented in textbooks used as fundamental literature in undergraduate and graduate-level courses. Although the assessment of unidimensionality is legitimate, CFA could similarly be applied to that end, and therefore we recommend its dissemination in the academic milieu.

On the other hand, the use of the Little Jiffy procedures is troublesome (albeit with genuinely exploratory objectives); this practice suggests a lax application of the technique − for example, when using the Varimax rotation, the researcher presupposes that it is reasonable to conceive that the factors obtained are orthogonal (i.e., uncorrelated); but in practice, it is unlikely that there is no correlation (r = 0) between factors that measure the same latent variable. This discussion should be addressed by faculty, undergraduate and graduate students, so that EFA applications are discussed critically and in depth, so that practitioners and researchers are properly trained in the analytical techniques.

From the previous observations and the results presented in Figure 2, a set of guidelines for future research in the area of Production and Operations emerge.

• 1^st End result: confirmatory methods (CFA and SEM) ought to be applied more frequently.

More than 20 years ago, Hinkin (1995, p. 982) recommended the widespread use of CFA. The results of the present research confirm this position, as can be seen from the comparison of national and international articles (first column in Figure 2, and first line of Table 2).

Table 2
Use of factor analysis by Production and Operations journal.

Type of application	National publications				International publications
	G&P	GEPROS	JOSCM	RPO	IJOPM	JBL	JOM	JSCM	POM	SCM
1 = CFA, SEM, PLS-PM	4	0	0	1	7	10	10	10	7	10
2 = EFA for Harman's test	0	0	1	0	4	4	4	8	1	6
3 = EFA to test unidimensionality	5	1	4	0	2	0	0	1	1	0
4 = Deductive EFA (Hinkin) and theory confirmation	1	1	0	1	1	0	0	0	0	4
5 = Deductive EFA (Hinkin) and theory destruction	1	1	0	2	0	0	0	0	0	0
6 = Exploratory EFA: common factor (PAF, ML...) + oblique rotation	0	0	0	0	0	0	1	0	0	0
7 = Exploratory EFA: principal components + Varimax (Little Jiff)	5	0	1	2	1	0	0	1	0	1
8 = Exploratory EFA: other combinations of extraction and rotation	0	0	0	1	0	0	0	0	0	0
9 = EFA with instrumental use	4	3	2	1	0	0	0	0	4	0
Total	20	6	8	8	15	14	15	20	13	21

• Source: Research data. Please note that the shaded lines indicate the questionable applications: that is, instances where the CFA is prescribed.

The third application (EFA to test unidimensionality) is one of the most used in Brazilian papers. It is important to note the incoherence of using an exploratory technique to test hypotheses, since it may be efficient in the evaluation of the convergent validity (via extracted variance), but that does not ensure that the discriminant validity is evaluated (which is specific to CFA).

The fifth application (destruction of theory) only occurrs in national articles, which is the worst case possible. Clearly, CFA or SEM should have been used in order to preserve the original conceptual model in these papers.

• 2^nd End result: necessary steps to minimize, detect and control method bias (common method bias).

When we compare the second application (One national article out of a total of 36; 27 international articles out of 61 in total), it becomes apparent that the treatment of the common method bias has not been common in national surveys. This problem was already raised, and produced a warning from the RAC Editorial (Kimura, 2015).

Even though international papers use the Harman test to detect the common method bias, authors such as Podsakoff et al. (2012) recommend the prevention of this bias. In cases where this is not feasible, one should use some method to control this bias, as suggested by Chin et al. (2013).

• 3^rd End result: The discussion of instances in which the use of EFA is appropriate, and those in which it is not, should be included in the content of textbooks. We note, nonetheless, that this issue remains in the border, and has been ignored by authors.

On the one hand, when EFA is taught strictly for exploratory use, the student will not be able to appreciate the other EFA applications. In this case, one needs to graps the proper structural and CFA modeling concepts (such as unidimensionality and Harman's test).

On the other hand, when teaching CFA and SEM, including EFA content seems to deviate from the main subject matter.

In any event, this “boundary” should be incorporated in books that discuss the development of scales in general, regardless of the fact that they focus specifically on EFA or CFA. Hence, all fields of knowledge in which measurement scales are used (and not just Production and Operations) will benefit from this development.

5 Final conclusions

The advancement of the theory in Production and Operations in the Brazilian context, as in any area, depends on the collaborative work from academia. The metaphor of “putting a brick in the building” is quite common, but it will remain a metaphor until the issues presented in this research are discussed, disseminated and taught to the entire scientific community.

In that sense, the field of Psychology has discussed and disclosed the problem of reproducibility of research at the national level (UnBCIÊNCIA, 2013) and international level (Nosek et al., 2015). In contrast, in the Production and Operations area, and in the Humanities, Social and Applied Social Sciences area, the use of confirmatory methods should be encouraged, in order to begin discussing the reproducibility of theory − which will in turn enable scientists to compare, test and expand concepts found in the literature.

Generally speaking, some recommended practices emerge from this work, for use by researchers and academics to promote the use of confirmatory methods:

• Include quantitative methods disciplines in graduate programs;

• Encourage the discussion of methodological procedures, and data analysis techniques in academic venues (congresses, symposia, etc.);

• Encourage the application of EFA in more advanced contexts, such as evaluation of the bias of data collection methods, which can be considered a fundamental requirement for publication in impact journals;

• Include confirmatory factor analysis and structural equations modeling in curricula;

• Use free and friendly software to do conduct analysis, such as:

  • − SmartPLS 2.0 for SEM with partial least squares estimation;

  • − Lavaan for CFA and SEM with estimation based on covariances (Beaujean, 2014; Rosseel, 2016).

• Editors should include guidelines in their editorials for authors and reviewers, as was done by Kimura (2015) regarding the common method bias test.

Appendix A List of national papers that were analyzed.

Gestão & Produção [17 papers]

Bortolotti, S. L. V., Moreira Jr., F. D. J., Bornia, A. C., Sousa Jr., A. F., & Andrade, D. F. (2012). Avaliação do nível de satisfação de alunos de uma instituição de ensino superior: uma aplicação da teoria da resposta ao item. Gestão & Produção, 19(2), 287–302.

Castelo, J. S. F., Coelho, A. F. M., & Cabral, J. E. O. (2014). Análise comparativa dos antecedentes e dimensões do brand equity entre marcas de fabricantes e marcas próprias no setor de vestuário. Gestão & Produção, 21(1), 19–32.

Conceição, S. H., Dourado, G. B., Baqueiro, A. G., Freire, S., & Brito, P. D. C. (2011). Fatores determinantes no disclosure em Responsabilidade Social Corporativa (RSC): um estudo qualitativo e quantitativo com empresas listadas na Bovespa. Gestão & Produção, 18(3), 461–472.

De Toni, D., Milan, G. S., & Reginato, C. E. R. (2011). Fatores críticos para o sucesso no desempenho de novos produtos: um estudo aplicado ao setor moveleiro da Serra Gaúcha. Gestão & Produção, 18(3), 587–602.

Ferreira, L. F. F., Oliva, F. L., Santos, S. A., Grisi, C. C. H., & Lima, A. C. (2012). Análise quantitativa sobre a mortalidade precoce de micro e pequenas empresas da cidade de São Paulo. Gestão & Produção, 19(4), 811–823.

Gomes, C. M., Kruglianskas, I., & Scherer, F. L. (2011). Gestão das fontes externas de informação: uma análise dos fatores que influenciam o desempenho inovador. Gestão & Produção, 18(4), 897–910.

Grohmann, M. Z., Battistella, L. F., & Lütz, C. (2014). Avaliação dos serviços de transporte aéreo brasileiro: análise da imagem e da atitude como antecedentes da intenção de uso. Gestão & Produção, 21(1), 215–227.

Lemos, B., & Joia, L. A. (2012). Fatores relevantes à transferência de conhecimento tácito em organizações: um estudo exploratório. Gestão & Produção, 19(2), 233–246.

Medina, R. M., & Crispim, S. F. (2010). Fatores determinantes no processo de decisão de investimentos em robotização na indústria brasileira de autopeças. Gestão & Produção, 17(3), 567–578.

Mendes, G. H. S., & Toledo, J. C. (2012). Explorando práticas do desenvolvimento de produtos em pequenas e médias empresas do setor de equipamentos médico-hospitalares. Gestão & Produção, 19(1), 103–117.

Oliveira Neto, G. C., Godinho Filho, M., Ganga, G. M. D., Naas, I. A., & Vendrametto, O. (2015). Princípios e ferramentas da produção mais limpa: um estudo exploratório em empresas brasileiras. Gestão & Produção, 22(2), 326–344.

Oliveira, B. R. B., Salazar, V. S., Crespo, P. M., Costa, C. S. R., & Kovacs, E. P. (2015). Estratégia competitiva em MPE’s: dimensões do processo e suas associações com as estratégias genéricas e funcionais. Gestão & Produção, 22(1), 119–132.

Sanchez, G. (2013) PLS Path Modeling with R. Trowchez Editions. Berkeley. Retrieved from: <http://gastonsanchez.com/PLS_Path_Modeling_with_R.pdf>.

Santos, A. B., & Antonelli, S. C. (2011). Aplicação da abordagem estatística no contexto da gestão da qualidade: um survey com indústrias de alimentos de São Paulo. Gestão & Produção, 18(3), 509–524.

Spessatto, G., & Beuren, I. M. (2013). Análise das diferenças na implantação do balanced scorecard nas maiores empresas da região sul do Brasil. Gestão & Produção, 20(2), 387–404.

Su, A. F. G., & Sampaio, M. (2013). Adaptação cultural e validação dos construtos: serviço logístico, satisfação e lealdade. Gestão & Produção, 20(3), 587–601.

Tacconi, M. de F. F. S., Lopes, F. D., Mói, A. L. R., & Tacconi Neto, E. A. (2014). A confiança interorganizacional nas compras. Gestão & Produção, 21(1), 199–214.

Tontini, G., & Zanchett, R. (2010). Atributos de satisfação e lealdade em serviços logísticos. Gestão & Produção, 17(4), 801–816.

Journal of Operations and Supply Chain Management [7 papers]

Borella, M. R. C., & Padula, A. D. (2010). Alignment between the supply, manufacturing and distribution strategies and business strategy. Journal of Operations and Supply Chain Management, 3(2), 44–60.

Filgueiras, J. L. M., Oliveira, M. G., Castro Neto, F. C., & Silva Filho, J. C. L. (2012). Analysis of CSR in midsize retail chains in Northeastern Brazil: application of the Quazi and O ’ Brien model. Journal of Operations and Supply Chain Management, 5(2), 54–68.

Martins, R. S., Xavier, W. S., Souza Filho, O. V., & Martins, G. S. (2010). Transport management in small and medium-sized enterprises in Brazil. Journal of Operations and Supply Chain Management, 3(1), 55–66.

Rashed, C. A. A., Azeem, A., & Halim, Z. (2013). Effect of information and knowledge sharing on supply chain performance: a survey based approach. Journal of Operations and Supply Chain Management, 3(2), 61–77.

Revilla, E., Prieto, I. M., & Rodriguez, B. (2011). Information technology and the ambidexterity hypothesis: an analysis in product development. Journal of Operations and Supply Chain Management, 4(2), 1–18.

Vachon, S. (2012). Technological capacity and environmental performance: a research note using country level data. Journal of Operations and Supply Chain Management, (Special Issue), 21–28.

Wanke, P. F., Correa, H. L., & Hijjar, M. F. (2010). Establishing the relationship between logistics complexity and supply chain objectives and decision areas in large companies operating in Brazil. Journal of Operations and Supply Chain Management, 3(1), 34–54.

Revista Produção Online [6 papers]

Artuso, A. R., & Chaves Neto, A. (2012). Identificação de fatores relevantes e construção de portfólio diversificado no mercado acionário brasileiro. Revista Produção Online, 12(4), 1002–1030. doi:10.1017/CBO9781107415324.004

Fernandes, R. B., Antonialli, L. M., Costa Filho, C. G., & Caixeta, R. P. (2015). A utilização de sistemas de ERP como antecedente da eficiência e eficácia (inter)organizacional: um estudo em dimensões estratégicas em pequenas e médias empresas. Revista Produção Online, 15(4), 1351–1376.

Moraes, R. O., & Laurindo, F. J. B. (2013). Relações entre o desempenho dos projetos de TI e a maturidade em gestão de projetos. Revista Produção Online, 13(1), 61–83. doi:10.1017/CBO9781107415324.004

Seidel, E. J., Lopes, L. F. D., Ansuj, A. P., & Zanella, A. (2011). Métodos estatísticos aplicados à avaliação da qualidade da matéria-prima e classificação dos fornecedores de uma indústria de laticínios. Revista Produção Online, 11(1), 3–28.

Stefano, N. M., & Casarotto Filho, N. (2012). Percepção dos consumidores: atributos considerados importantes nas embalagens. Revista Produção Online, 12(3), 657–681.

Teles, F., & Silva, R. M. (2014). Avaliação da satisfação dos usuários de sistemas ERP nas médias empresas de confecção do vestuário de Fortaleza. Revista Produção Online, 14(2), 533–559.

GEPROS. Gestão de Produção, Operações e Sistemas [6 papers]

Aharonovitz, M. C. S., & Vieira, J. G. V. (2014). Proposta de modelo multicritério para seleção de fornecedores de serviços logísticos. GEPROS. Gestão de Produção, Operações e Sistemas, 9(1), 9–26.

Buttner, A., & Morano, R. S. (2013). Vantagem competitiva através do Planejamento Estratégico: avaliação de firmas brasileiras de autopeças. GEPROS. Gestão de Produção, Operações e Sistemas, 8(3), 25–37.

Cavalheiro, E. A., Tavares, C. E. M., Ferreira, A. P. A. L., Santos, R. A., & Stedile, C. S. M. (2014). Modelo europeu de satisfação: um estudo de caso com discentes de uma instituição de ensino superior. GEPROS. Gestão de Produção, Operações e Sistemas, 9(1), 131–142.

Graeml, A. R., Peinado, J., & Glaser-Segura, D. (2013). Fatores culturais: uma análise sobre competitividade e coletivismo na indústria de países emergentes. GEPROS. Gestão de Produção, Operações e Sistemas, 8(3), 9–23.

Menezes, L., & Borchardt, M. (2010). Análise dos fatores que determinam a percepção de qualidade e o uso de serviços de atendimento presenciais. GEPROS. Gestão de Produção, Operações e Sistemas, 5(4), 29–45.

Souza, A. A., Alves, A. de S., Silva, D., Alves, L. G., Avelar, E. A., & Lara, A. L. (2015). Avaliação de sistemas de informações com foco na satisfação dos usuários: um estudo desenvolvido na região metropolitana de Belo Horizonte - MG. GEPROS. Gestão de Produção, Operações e Sistemas, 10(4), 69–84.

Appendix B List of international papers that were analyzed (24 out of 61).

This list contains four articles from each publication that was analyzed. The complete list (61 papers) can be obtained with the first author.

Production and Operations Management

Abrahams, A. S., Fan, W., Wang, G. A., Zhang, Z., & Jiao, J. (2015). An integrated text analytic framework for product defect discovery. Production and Operations Management, 24(6), 975–990.

Chandrasekaran, A., Linderman, K., & Schroeder, R. (2015). The role of project and organizational context in managing high-tech R&D projects. Production and Operations Management, 24(4), 560–586.

Li, X., Li, Y., Cai, X., & Shan, J. (2016). Service Channel Choice for Supply Chain: Who is Better off by Undertaking the Service? Production and Operations Management, 25(3), 516–534.

Yu, S., Mishra, A. N., Gopal, A., Slaughter, S., & Mukhopadhyay, T. (2015). E-Procurement Infusion and Operational Process Impacts in MRO Procurement: Complementary or Substitutive Effects? Production and Operations Management, 24(7), 1054–1070.

Journal of Operations Management (JOM)

Liu, S. (2015). Effects of control on the performance of information systems projects: The moderating role of complexity risk. Journal of Operations Management, 36, 46–62.

Choo, A. S., Nag, R., & Xia, Y. (2015). The role of executive problem solving in knowledge accumulation and manufacturing improvements. Journal of Operations Management, 36, 63–74.

Pagell, M., Klassen, R., Johnston, D., Shevchenko, A., & Sharma, S. (2015). Are safety and operational effectiveness contradictory requirements: The roles of routines and relational coordination. Journal of Operations Management, 36, 1–14.

Tenhiälä, A., & Helkiö, P. (2015). Performance effects of using an ERP system for manufacturing planning and control under dynamic market requirements. Journal of Operations Management, 36, 147–164.

International Journal of Operations & Production Management

Russell, R. S., Johnson, D. M., & White, S. W. (2015). Patient perceptions of quality: analyzing patient satisfaction surveys. International Journal of Operations and Production Management, 35(8), 1158–1181.

von Haartman, R., & Bengtsson, L. (2015). The impact of global purchasing and supplier integration on product innovation. International Journal of Operations and Production Management, 35(9), 1295–1311.

Yam, R. C. M., & Chan, C. (2015). Knowledge sharing, commitment and opportunism in new product development. International Journal of Operations and Production Management, 35(7), 1056–1074.

Zhang, M., Zhao, X., Lyles, M. A., & Guo, H. (2015). Absorptive capacity and mass customization capability. International Journal of Operations and Production Management, 35(9), 1275–1294.

Journal of Supply Chain Management

Cai, S., & Yang, Z. (2014). The role of the guanxi institution in skill acquisition between firms: a study of Chinese firms. Journal of Supply Chain Management, 50(4), 3–24.

Handfield, R. B., Cousins, P. D., Lawson, B., & Petersen, K. J. (2015). How can supply management really improve performance? A knowledge-based model of alignment capabilities. Journal of Supply Chain Management, 51(3), 3–17.

Ralston, P. M., Blackhurst, J., Cantor, D. E., & Crum, M. R. (2015). A structure-conduct-performance perspective of how strategic supply chain integration affects firm performance. Journal of Supply Chain Management, 51(2), 47–64.

Whipple, J. M., Wiedmer, R., & Boyer, K. K. (2015). A Dyadic investigation of collaborative competence, social capital, and performance in buyer-supplier relationships. Journal of Supply Chain Management, 51(2), 3–21.

Supply Chain Management: An International Journal

Flynn, B. B., Huo, B., & Zhao, X. (2010). The impact of supply chain integration on performance: A contingency and configuration approach. Supply Chain Management: An International Journal, 28(1), 58–71.

Lee, S.-Y. (2015). The effects of green supply chain management on the supplier’s performance through social capital accumulation. Supply Chain Management: An International Journal, 20(1), 42–55.

Tanco, M., Jurburg, D., & Escuder, M. (2015). Main difficulties hindering supply chain performance: an exploratory analysis at Uruguayan SMEs. Supply Chain Management: An International Journal, 20(1), 11–23.

Yu, W., Chavez, R., Feng, M., & Wiengarten, F. (2014). Integrated green supply chain management and operational performance. Supply Chain Management: An International Journal, 19(5/6), 683–696.

Journal of Business Logistics

Adams, F. G., Richey, R. G., Autry, C. W., Morgan, T. R., & Gabler, C. B. (2014). Supply chain collaboration, integration, and relational technology: How complex operant resources increase performance outcomes. Journal of Business Logistics, 35(4), 299–317.

Gattiker, T. F., Carter, C. R., Huang, X., & Tate, W. L. (2014). Managerial commitment to sustainable supply chain management projects. Journal of Business Logistics, 35(4), 318–337.

Gligor, D. M. (2014). A cross-disciplinary examination of firm orientations’ performance outcomes: The role of supply chain flexibility. Journal of Business Logistics, 35(4), 281–298.

Schoenherr, T., Modi, S. B., Talluri, S., & Hult, G. T. M. (2014). Antecedents and Performance Outcomes of Strategic Environmental Sourcing: An Investigation of Resource-Based Process and Contingency Effects. Journal of Business Logistics, 35(3), 172–190.

Acknowledgements

The present work was carried out with the support of CNPq, National Council of Scientific and Technological Development - Brazil. Process #: 409582 / 2013-1. The first author is a Fellow of CNPq - Brazil.

• Financial support: Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq): Process 409582/2013-1 and 311477/2015-1. The first author is CNPq PQ Scholarship – Brazil.

Referências

• Aranha, F., & Zambaldi, F. (2008). Análise fatorial em administração São Paulo: Cengage Learning.
• Beaujean, A. A. (2014). Latent variable modeling using R: a step-by-step guide New York: Routledge.
• Bido, D. S. (2014). Escalas como ferramentas de diagnóstico e gestão: que peso dar aos dados (análise fatorial exploratória) e que peso dar à teoria e pesquisas anteriores (análise fatorial confirmatória)? In Anais do 38º EnANPAD Rio de Janeiro: ANPAD.
• Chin, W. W., Thatcher, J. B., Wright, R. T., & Steel, D. (2013). Controling for common method variance in PLS analysis: the measured latent marker variable approach. In H. Abdi, W. W. Chin, V. E. Vinzi, G. Russolillo & L. Trinchera (Eds.). New perspectives in partial least squares and related methods (pp. 231-239). New York: Springer.
• Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3. ed.). New Jersey: Lawrence Erlbaum Associates.
• Conway, J. M., & Huffcutt, A. I. (2003). A review and evaluation of exploratory factor analysis practices in organizational research. Organizational Research Methods, 2(6), 147-168. http://dx.doi.org/10.1177/1094428103251541
  » http://dx.doi.org/10.1177/1094428103251541
• Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 1-9.
• Fabrigar, L. R., Wegener, D. T., Maccallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 3(4), 272-299. http://dx.doi.org/10.1037/1082-989X.4.3.272
  » http://dx.doi.org/10.1037/1082-989X.4.3.272
• Fávero, L. P., Belfiore, P., Silva, F. L., & Chan, B. L. (2009). Análise de dados: modelagem multivariada para tomada de decisões (1. ed.). Rio de Janeiro: Elsevier.
• Guide, V. D. R., Jr., & Ketokivi, M. (2015). Notes from the editors: restructuring the Journal of Operations Management. Journal of Operations Management, 38, 1-86. http://dx.doi.org/10.1016/S0272-6963(15)00073-X
  » http://dx.doi.org/10.1016/S0272-6963(15)00073-X
• Hair, J. F., Jr., Babin, B., Money, A. H., & Samouel, P. (2005). Fundamentos de métodos de pesquisa em administração Porto Alegre: Bookman.
• Hair, J. F., Jr., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7. ed.). Upper Side River: Prentice Hall.
• Hinkin, T. R. (1995). A review os scale development practices in the study of Organizations. Journal of Management, 5(21), 967-988. http://dx.doi.org/10.1177/014920639502100509
  » http://dx.doi.org/10.1177/014920639502100509
• Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1), 104-121. http://dx.doi.org/10.1177/109442819800100106
  » http://dx.doi.org/10.1177/109442819800100106
• Hurley, A. E., Scandura, T. A., Schriesheim, C. A., Brannick, M. T., Seers, A., Vanderberg, R. J., & Williams, L. J. (1997). Exploratory and confirmatory factor analysis: guidelines, issues and alternatives. Journal of Organizational Behavior, 18(6), 667-683. http://dx.doi.org/10.1002/(SICI)1099-1379(199711)18:6<667::AID-JOB874>3.0.CO;2-T
  » http://dx.doi.org/10.1002/(SICI)1099-1379(199711)18:6<667::AID-JOB874>3.0.CO;2-T
• Jöreskog, K. G. (2007). Factor analysis and its extensions. In R. Cudeck & R. C. MacCallum (Eds.), Factor analysis at 100: historical developments and future directions (pp. 47-77). New Jersey: Lawrence Erlbaum Associates.
• Kaiser, H. F. (1970). A second generation Little Jiffy. Psychometria, 4(35), 401-415. http://dx.doi.org/10.1007/BF02291817
  » http://dx.doi.org/10.1007/BF02291817
• Kimura, H. (2015). Editorial. Revista de Administração Contemporânea, 3(19), 1-5.
• Malhotra, N. (2012). Pesquisa de marketing: uma orientação aplicada (6th ed.). Porto Alegre: Bookman.
• Mantovani, D. M. N., & Bido, D. S. (2015). Análise Fatorial na Pesquisa em Administração: exploratória ou confirmatória? Eis a questão. In Anais do 39º EnANPAD Rio de Janeiro: ANPAD.
• Nosek, B. A., Aarts, A. A., Anderson, C. J., Anderson, J. E., & Kappes, H. B. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. PMid:26315443. http://dx.doi.org/10.1126/science.aac4716
  » http://dx.doi.org/10.1126/science.aac4716
• Osborne, J. W. (2014). Best practices in exploratory factor analysis Louisville: CreateSpace Independent Publishing Platform. E-book.
• Pasquali, L. (2012). Análise fatorial para pesquisadores Brasília: LabPAM.
• Pett, M. A., Lackey, N. R., & Sullivan, J. J. (2003). Making sense of factor analysis: the use of factor analysis for instrument development in health care research Thousand Oaks: Sage.
• Podsakoff, P. M., Mackenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63(1), 539-569. PMid:21838546. http://dx.doi.org/10.1146/annurev-psych-120710-100452
  » http://dx.doi.org/10.1146/annurev-psych-120710-100452
• Podsakoff, P. M., Mackenzie, S. B., Lee, J. Y., & Podsakoff, M. P. (2003). Common method biases in behavioral research: a critical review of the literature and recommended remedies. The Journal of Applied Psychology, 5(88), 879-903. PMid:14516251. http://dx.doi.org/10.1037/0021-9010.88.5.879
  » http://dx.doi.org/10.1037/0021-9010.88.5.879
• Rosseel, Y. (2016). The lavaan tutorial Belgium: Department of Data Analysis, Ghent University. Recuperado em 9 de agosto de 2016, de http://lavaan.ugent.be/tutorial/tutorial.pdf
  » http://lavaan.ugent.be/tutorial/tutorial.pdf
• Sanchez, G. (2013). PLS Path Modeling with R Berkeley: Trowchez Editions. Recuperado em 9 de agosto de 2016, de http://gastonsanchez.com/PLS_Path_Modeling_with_R.pdf
  » http://gastonsanchez.com/PLS_Path_Modeling_with_R.pdf
• UnBCIÊNCIA. (2013). Consórcio replica 13 efeitos experimentais de estudos em Psicologia Recuperado em 12 de outubro de 2017, de https://www.unbciencia.unb.br/humanidades/97-psicologia/78-consorcio-replica-13-efeitos-experimentais-de-estudos-em-psicologia
  » https://www.unbciencia.unb.br/humanidades/97-psicologia/78-consorcio-replica-13-efeitos-experimentais-de-estudos-em-psicologia
• Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: a content analysis and recommendations for best practices. The Counseling Psychologist, 6(34), 806-838. http://dx.doi.org/10.1177/0011000006288127
  » http://dx.doi.org/10.1177/0011000006288127
• XLSTAT. (2017a). Consumer satisfaction analysis in Excel with PLSPM Recuperado em 12 de outubro de 2017, de https://help.xlstat.com/customer/es/portal/articles/2239701-applying-a-plspm-model-to-analyze-consumer-satisfaction?b_id=9283
  » https://help.xlstat.com/customer/es/portal/articles/2239701-applying-a-plspm-model-to-analyze-consumer-satisfaction?b_id=9283
• XLSTAT. (2017b). PLS Path Modelling Recuperado em 12 de outubro de 2017, de https://www.xlstat.com/en/solutions/features/pls-path-modelling
  » https://www.xlstat.com/en/solutions/features/pls-path-modelling

Publication Dates

• Publication in this collection
  30 Oct 2017
• Date of issue
  Apr-Jun 2018

History

• Received
  09 Aug 2016
• Accepted
  24 Mar 2017