Measures in organizational behavior: why don’t we use big data and analytics? não usamos big data e analytics?

Tradeoff analysis, between generalizability, precision, and realism, guides methodological choices in organizational behavior. These methodological choices were systematically reviewed in Brazilian articles and books (1996-2017) to answer the following question: why are there no Brazilian studies on organizational behavior that use big data or analytics? Among 1062 research articles on organizational behavior, published in 19 psychology and business journals, 68% used scales, and only 10% used observation. Observation was often unstructured and supported other methods. The focus was on “saying,” instead of “doing”. Big data and analytics have the potential to simultaneously reach generalizability, precision, and realism and may pave the way for new conclusions. Additionally, these methods could transform research in organizational behavior.

The colossal and varied volume of data currently available is revolutionizing the workplace (Kopp, Howaldt, & Schultze, 2016;Pinzone et al., 2017). Techniques employing big data can identify complex signals by using analytics (machine learning, statistical learning, and data mining) (Mortenson, Doherty, & Robinson, 2015). For over a decade, these techniques are already used in Brazil in fields such as logistics (Aires, Almeida, & Silveira, 2019;Ferreira et al., 2016;Wu & Cunha, 2008), marketing (Carnasciali & Delazari, 2007;Cunha, Silva, & Chaves, 2018;Santos & Carvalho, 2016), and finance (Ferreira & Costa, 2017;Lucio, 2009;Porto, 2017). The Work and Organizational Psychology (WOP) subarea should use more methods featuring these technological innovations, recommend Grimm, Jacobucci, and McArdle (2017). Why aren't there Brazilian studies in WOP that use big data or analytics? This study aims to answer this question. It reviews the methods used in Organizational Behavior (OB) and analyses the use of scales, which is the predominant method, and behavior observation, which may be the most affected method by the abovementioned technological innovations.
Geopolitical and economic changes and technological innovations have promoted significant work and organizational changes; these changes have impacted WOP, which has expanded and become solid. As a subarea of knowledge, WOP analyzes human behavior within work and organization contexts. It can also be understood as a field of application wherein it solves practical problems. They are related to the interactions between human behavior, work, and organizations, and to the practices of these organizations, which have already incorporated several innovations of the so-called Information and Communications Technologies (ICT).
Studying the mutual influence between individuals and organizations is a challenge that requires investigations at the individual, team, and organizational levels. The fields of anthropology, sociology, administration, and psychology aimed at studying this influence based on their theoretical and methodological frameworks or developed other frameworks. They divided this influence into concepts and relationships, that were analyzed from different perspectives and used different methodological approaches. The broad and unsystematic investigation of the phenomena caused great fragmentation (Schein, 2015). Thus, several WOP subcultures emerged, each with its jargon and procedures. Each subculture developed an intellectual silo distant from a main theory and other silos, argued this author. Another impact was a possible disconnection between what scholars produced (knowledge area) and what professionals applied (field of application). Research studies are designed to obtain as much validity as possible and not necessarily to achieve maximum impact. On the other hand, professionals may twist the scholars' findings to justify their interventions.
The application of WOP began with regular use of measures without evidence of validity in Brazil and Latin American (Borges-Andrade, Renteria, Pérez, & Toro, 2018). The number of professionals in this field of application and the postgraduate programs, constituting the subarea of knowledge, increased sharply in Brazil, suggested these authors. They argued that the accelerated growth of the subarea of knowledge lead to many publications and the use of diverse approaches. Undergraduate programs offer few courses on WOP. As for professional internships, they focus on apprehension and application and are often not mandatory. This means that no skills are developed related to building knowledge and evaluating interventions. This scenario represents an inadequate academic training, making professional practice (field of application) defective and disconnected from academic production (subarea of knowledge).
Organizational behavior is a field grounded on psychology, sociology, and economics and is a relevant part of the scientific production of WOP. According to Siqueira (2009), its objects are organizational activities, that merge in three distinct and connected levels, namely, macro (organizations), meso (teams and groups), and micro (individual) levels. Organizational behavior adopted the idea that the method to evolve a theory must divide the human behavior into multiple components, measure these components and, through factorial analysis, unveil the underlying variables (Schein, 2015). A field's production that consists in only one category of methods is prone to measuring errors; it could even end up prioritizing some errors.
Organizational behavior may be analyzed based on Burrell and Morgan (1979), where paradigms are divided into two axes, namely, concepts of society and social science. Moreover, the latter separates its approaches into subjectivist and objectivist. Each of these approaches has its own ontology, epistemology, view of human nature, and methodology. Therefore, this division implies that the paradigms from which the approaches originate define the method they use to investigate phenomena. Subjectivists base their ideas on the assumption that an individual's external world is not built of real structures but denominations and conventions. This aspect causes the search for patterns and regularities to be excluded, as each person's reality is unique. Thus, diaries and biographies, as methods of investigations, aid in understanding the social world by holistically comprehending the reality. Objectivists base their ideas on the assumption that an individual's external world is composed of tangible and relatively unchangeable structures. This aspect makes them seek patterns and regularities that enable them to explain and predict phenomena, for which they employ controlled scientific tests, standardized tools, and quantitative data analysis (Corradi, Marcon, Loiola, Kanan, & Vieira, 2016).
Another paradigmatic division organizes WOP into three axes, namely, behavioral, subjectivity, and clinical (Bendassolli, Borges-Andrade, & Malvezzi, 2010). The behavioral axis is based on post-positivism and seeks objectivity; even though it is understood that these functions cannot be fully achieved. The main influences of this axis are behaviorism, neobehaviorism, and sociocognitivism, which originally used experimental methods. The subjectivity axis uses the critical paradigm, which opposes post-positivism in its planning, execution, and interpretation of research results. Its main influences are the socio-historical and institutional approaches and post-structuralism. It uses methods and techniques that focus on analyzing the phenomenon, thus eliminating comparability and generalization of the results. The clinical axis also considers the critical paradigm as a theoretical paradigm and adopts qualitative methodologies for the analysis of the individual's background. The third axis includes ergology, psychodynamics of work, clinical sociology, and the clinic of activity (Bendassolli et al., 2010).
Although paradigms influence methodological choices, availability of tools may have a greater influence on the methodological choices. In other words, if an ideal tool existed, that would provide continuous access to a phenomenon, measuring all its variables without intervention, then all approaches could employ this tool.
The dilemmatic viewpoint of psychology research begins with the tradeoff analysis made in methodological choices, considering the lack of an ideal tool. This view indicates that all methods have errors and that researchers need to learn to deal with these errors. According to this viewpoint, there are three factors (or horns), namely, generalizability, precision, and realism. It would not be possible to satisfy (or grab) all three horns simultaneously. Thus, each researcher makes methodological decisions that favor one or two of the abovementioned horns; however, the researcher's decisions can never simultaneously favor all the three horns. For example, surveys favor generalizability and exclude precision and realism. On the contrary, ethnographic research favors realism and exclude precision and generalizability. As no method grabs all the three horns, methodological diversification and colossal volume of research to study the same phenomenon are the derived solutions (McGrath, 1981). Therefore, irrespective of the epistemic positioning of the study's authorship, it is subjected to tool-related restrictions; these tool-related constraints may be shifted by technological innovations.
Scientific research in OB is almost always based only on what an individual says, through qualitative or quantitative analyses. This predominant form of access to what is "said" could question the validity of the findings (Aguinis, Pierce, Bosco, & Muslin, 2009). Only 12.4% of Brazilian studies reported behavior observation as a method, and most observations are also focused on what the individual says, despite the claim to consider what people do. Thus, they do not incorporate fundamental elements to appropriately understand the phenomenon (Mourão, Bastos, Passos, & Oliveira, 2016). Studies on performance, commitment, and organizational citizenship refer directly to organizational behaviors, but these aspects are investigated almost exclusively by what individuals say, as indicated by Mourão et al. (2016).
The response to scales model, particularly those referring to behaviors, indicates steps that the respondent supposedly follows when answering an item, namely, (i) understand the question, (ii) understand the behavior of interest, (iii) correctly identify the period of interest, (iv) remember this period to identify all relevant incidents of the behavior, (v) correctly date these recalled incidents, (vi) add them correctly, (vii) place their quantity into the proposed response alternatives, and (viii) answer the question (Schwartz & Oyserman, 2001). Each step involves the possibility of making errors and its external validity cannot be attested even if the measure is internally consistent. To understand OB constructs, considering their inherent complexity and the difficulty to categorize them in a validly and reliably manner, a triangulation of methods is required where different ways of investigating "doing" and "saying" should be employed to ensure that the information is used together. Studies should focus less on responses based only on "saying" and seek ways to access "doing", suggested Mourão et al. (2016). Consequently, this aspect would probably involve direct observations of behaviors.
Historical explanations, technological limitations, and operational difficulties are involved in the method of directly observing human behaviors in organizations. The first difficulty is to convince these organizations to allow to conduct observations related to their employees' behaviors. The second difficulty involves the reliability of the collected data, as the observer's presence affects the performance/behaviors of those being observed. The third difficulty is tradeoff analysis as this method requires considerable time and more than one observer to access even a slight change in one's behavior while at work. The fourth difficulty is that data consistency is rarely achieved, as observers need to be well trained. Finally, some organizational context variables cannot be controlled, thus leading to low reproducibility (Mourão et al., 2016). Observation methods were almost entirely replaced by surveys in the behavior axis and are used in an unsystematic manner in the other two axes to support the investigation of the individual's context. Unfortunately, a subarea of knowledge that aims to examine, predict, and correlate behaviors has supposedly abandoned direct access to them. On the bright side, there is a new methodological option.
Analyzing "doing" in organizations may be accomplished in several ways. Organizations nowadays have huge amounts of data (big data) available, although they can rarely analyze it themselves. One of the areas that seek to analyze big data is called analytics, which is a field closely related to computer science and engineering that employs quantitative methods to organize, analyze, and draw conclusions from huge raw databases (Mortenson et al., 2015). The combination of big data and analytics may identify complex signals in the data through analysis based on statistical techniques (machine learning, statistical learning, and data mining) (Grimm et al., 2017). This combination has revolutionized areas such as economics and applied social sciences.
The application of analytics for human resources is called human resources analytics or people analytics. People analytics has obtained great popularity in recent years and is being considered not only a trend but a reality (Deloitte University Press, 2017). The big data available in organizations is raw, unstructured, and derived from various sources such as emails, social media communications, web searches, images, and locations. These data, which are often behaviors, need to be handled and transformed into indicators so that they may be analyzed (Isson & Harriott, 2016). This task should be addressed to WOP professionals because they have the necessary theoretical knowledge. However, the lack of skills to understand data and manipulate databases makes it impossible for them to develop the task . The biggest challenge of people analytics is the integration of data to draw statistically valid and context-appropriate conclusions from data processing and analysis. This subarea lacks methods that integrate the conceptual knowledge of WOP into the new ways of analyzing data (Angrave, Charlwood, Kirkpatrick, Lawrence, & Stuart, 2016).
Big data and analytics have the potential to eliminate, or at least reduce, dilemmas such as that of the three abovementioned horns, previously regarded as inherent in the investigation of phenomena in social sciences. Generalizability, precision, and realism could be achieved simultaneously, and scientific research would become predictive rather than being explanatory (Chang, Kauffman, & Kwon, 2014). The main trends in using big data in WOP involve people selection, performance assessment, and identification of relationships networks (Guzzo, Fink, King, Tonidandel, & Landis, 2015). Considering this new opportunity and the dilemma of the three horns, this study analyzes the methods used in Brazilian publications on OB.

Method
The Brazilian publications on the OB field were analyzed using the review method described by Bastos, Maia, Rodrigues, Macambira, and Borges-Andrade (2014); Cantal, Borges-Andrade, and Porto (2015) Graduate students, previously trained in analysis of research reports on OB, examined and classified the articles in pairs, which were subsequently reviewed by another pair. The articles were classified into 17 major OB themes: affection, learning, attitude when facing changes, organizational climate, cognition, skills, escape and avoidance behaviors, psychological contracts, creativity and problem-solving, organizational culture, productive performance, deviations in behavior, well-being and health, social interactions, motivation, meanings, senses and identity at work, and decision-making. In each article, information was collected on the nature and purpose of the study, characteristics of the research design, sampling, data source, type of measurement, data analysis procedure, sector, and segment of the economy. A total of 1,062 articles, which were published between 1996 and 2017, met the described criteria. The reported data collection tools were classified as scale or questionnaire, interview, observation, documentary analysis, and test or exam.
Measures that focused on OB were also collected from four books (Abbad et al., 2009;Puente-Palacios & Peixoto, 2015;Siqueira, 2009Siqueira, , 2014. These books, produced during the abovementioned period, aimed to provide scales with good evidence of validity and previously published in research reports in articles, theses, and dissertations. This provision aimed to enable WOP professionals to become aware of the tools they could use in their work. This strategy was used because most of these professionals do not usually have the skills to find research reports on measures, and when they have access to them, they rarely understand the technical language used in such reports. Each of those measures (n = 87), in each of the mentioned books, was classified according to the same 17 themes in which the previous articles were classified. Regarding the behaviors that were objects of the measurements, all articles reporting the use of the method were reanalyzed and classified according to the paradigmatic axis of the study, the use of quantitative analysis, the prior definition of behaviors or constructs to be observed, the systematicity of observation, the researcher's participation, and the way in which the results were used in the research.

Results
There were two periods of acceleration in the dissemination of the Brazilian production on OB articles:  One of the milestones to produce scales may have been the article by Siqueira (2002). She analyzed OB measurements until 2002 and highlighted the lack of standardization in the measure-building process and the lack of measures with good evidence of validity. She emphasized that theoretical diversity made it difficult to define constructs to be analyzed and analysis procedures that captured all the complexity of the phenomena. Ultimately, she argued that this aspect culminated in the use of scales with low evidence of validity or scales that were built for other contexts. There has been significant growth in the scale production since 2002. However, this growth does not represent a methodological or procedural unification, as proposed by the mentioned author, but rather an even greater fragmentation of the subarea. As the paradigmatic axes of OB began to increasingly accept scales as a measuring tool, each of them developed tools to measure their constructs, rather than seeking unification. In most studies (87%) where there prevails a construction of a Estud. psicol. I Campinas I 38 I e180157 2021 scale, the goal is only the construction of the scale itself. On comparing the scales published in books and articles, the production was similar from 1996 to 2007 but has become very discrepant in the past decade. The production of scales reported in articles in the past decade was more than double the disclosure of scales in books ( Figure 2). Less organizational behavior scale production was performed in the following themes: decisionmaking and judgments, creativity and problem-solving, escape and avoidance behaviors, and attitude when facing changes (Figure 3). Among the themes in which production was numerically relevant, those that were balanced for the two means of the disclosure include meanings, senses, and identity at work; motivation; social interactions; deviations in behavior; skills; learning; and affection. The themes that followed the pattern of greater dissemination of scales in journals than in books include well-being and health, productive performance, and organizational climate. Finally, the themes that were opposite to the standard, with more reports in books than in journals, include organizational culture, psychological contracts, and cognition.

Quantity
The data presented here refer to the volume of reports on OB scales and not on their use in the WOP applied field. For example, organizational climate scales are among the most used by WOP professionals, although it is known that many of them have never undergone any analysis of validity evidence. They are the product of "home-made" solutions or they were directly imported by consulting companies that sell diagnosis of organizational climate. The number of climate scales in books (0.9%) is small concerning the total number of scales in books, being even less representative than the number of scales in articles (3.8%).  The use of observation as a method of data collection in studies on OB is scarce (8%). On the contrary, scales are present in most of these studies (68%). The representativeness of the use of scales varies greatly according to the OB theme. For example, almost all studies (85%) on psychological contracts use a scale, and less than half of the studies (43%) on meanings, senses, and identity at work use scales (Figure 4). Despite this variation, all themes have a significant number of studies where scales are used. On the contrary, the use of the observation method varies little among the OB themes. Even for themes that refer directly to behaviors, such as escape and avoidance behaviors and deviations in behaviors, few or no studies have reported the use of the observation method.
A more in-depth analysis of the 81 articles on OB, where the observation method was reported, reveals some possible patterns. The first pattern is that almost none (4%) of them presented a quantitative analysis of the observation data. Although some studies used combinations of methods such as field diary and Bardin's categorical analysis, which would allow quantitative data analysis, none described such analyses. Only 3 out of the 81 articles used quantitative analysis. They represent an exception to the rule that seems to have dominated the use of observation in the OB studies found in the extant literature review. None of the 1,062 studies used big data or analytics. Most of the articles describing the use of the observation method come from the subjectivity axis, and very few of them come from the clinical axis (Table 1). About the observed procedures, most studies use the exploratory method, without defining the behaviors to be observed or the construct of interest for which expressions should be observed through behaviors. Moreover, there is a great variety of methods that use observation in an exploratory manner (e.g., ethnography); however, these methods seek to understand a specific reality, and their results cannot be generalized. As for the results of the observation, in the behavioral axis, in a quarter of the cases, they are not even cited. When cited, they often supplemented the creation of questionnaires or scales. That is, the findings obtained through observation only served at an intermediate stage of a research process, in which a survey on "doing" would be replaced by the investigation on "saying."

Discussion
The Brazilian publications on OB, mainly from the behavioral axis, have scales as a predominant method of data collection. Several articles are published only with the report of the validation of a scale. Considering that professionals in the field of application of WOP rely on books and not articles , many validated scales are rarely employed or not employed at all. The constructs for which the scales were created may not necessarily be useful to professionals in this area, as a significant discrepancy prevails between the themes of the scales published in journals and books.
Further, scales favor generalizability, but they may lack precision and realism. They are subjected to the reliability problems, highlighted by Schwartz and Oyserman (2001), and validity problems, as they are based solely on what the individual says (Mourão et al., 2016). Several constructs, despite being investigated in various contexts, are not necessarily true to reality. A triangulation of methods is required to reduce the problems associated with measurements.
The low representativeness of the observation method is unsettling. Studies on OB do not seem to analyze what people do but rather what they say. This aspect can be explained by the high cost associated with using this method and the organizations' resistance to make room for the method to be used in their environments (Mourão et al., 2016). It can be a method of high precision, but with low generalizability, because it is used in small samples, and low realism, considering the researcher's interference. All the listed problems apply to the classic methods of observation; however, when classic methods of observation are performed using behaviors in virtual environments or technological resources to record behaviors, all the above mentioned issues are inapplicable. New ICT allow researchers to measure many behaviors without interference.
The behaviors recorded through ICT present two new challenges, namely, data handling and analysis (Guzzo et al., 2015). By combining the WOP's existing theoretical framework, big data, and analytics, a considerable amount of data may be accessed, which further can be analyzed and interpreted according to the existing theories. This aspect could lead to better generalizations and new conclusions that are likely to be closer to what professionals really do at work.
Behavior records stored in the database may or may not be accessible through virtual media (e.g., access to websites and recording through a computer's camera). Some kinds of stored information are much more complex than the mentioned ones and go beyond human conscious perception (e.g., pupil dilation, galvanic skin response, body temperature, and facial micro expressions). As for data analysis, the benefits of big data can come from colossal data, great statistical power, or new techniques. Large databases facilitate in unveiling patterns that are not observable on smaller samples. One application of this benefit is the identification of personality traits based on behaviors in social networks (Youyou, Kosinski, & Stillwell, 2015).
New data analysis techniques include textual mining and mapping of relationship networks, which draw entirely new conclusions. The application of these benefits to WOP is still in the early stages, but some examples are already available. For instance, data mining was used to improve the effectiveness of personnel selection by Aghaee and Aghaee (2016) and Azar, Sebt, Ahmadi, and Rajaeian (2013). An algorithm that identifies relevant information, in resumés and job openings, and relates applicants to job openings was designed by Paoletti, Martinez-Gil, and Schewe (2015). Another algorithm, which predicts employee performance evaluation, aimed at promoting performance improvement, was developed by Kirimi and Moturi (2016). A software called CORPNET, which analyzes formal and informal relationships to build relationship networks and identify power and organizational structures, was developed by Liu and Moskvina (2016). Eye movements, notes, logs, and facial recognition to identify self-regulatory processes during learning were used by Azevedo et al. (2013).
Big data and analytics, in a dilemmatic situation, have the potential to change the tradeoff of methodological choices. Achieving generalizability, precision, and realism simultaneously is possible as information comes from tools that are less intrusive and store large amounts of data at low cost (Chang et al., 2014). In a paradigmatic view, they are closer to the axis of behavior or objectivist approach. However, the consolidation of the application of scales and the standardization of data analysis procedures seem to hinder the search for innovations by this axis. Behavior observation is used more frequently by the subjectivist axis and approach, which could take advantage of innovations to improve existing techniques. However, to use the new techniques, training is required for handling and analyzing large databases, which is an uncommon aspect in the area of WOP. Considering this scenario, the evolution of WOP in Brazil may be hindering the introduction of new technologies in the area.
We reiterate the following question: In Brazil, why don't we find studies that use big data or analytics in WOP? The methodological choices of WOP, based on the paradigms from which each axis is derived, led to the construction of methodological frameworks that do not systematically observe human behaviors. Because big data and analytics rely on raw records, the lack of methods to systematically analyze behaviors prevents their use. It is thus necessary to review the methodological choices made in each paradigmatic axis and subarea of WOP to overcome this shortcoming and finally develop new methodological frameworks. There are several application possibilities for big data and analytics, and we expect them to completely transform the methods used in WOP.