Acessibilidade / Reportar erro

The measurement of social impact and opportunities for research in business administration This short article is based on the proposal for a Professional Development Workshop at the Academy of Management Conference in 2017, jointly organized with Sandro Cabral and Aldo Musacchio.

Abstract

In the last decades, management scholars have examined ways through which firms can build profitable operations while at the same time addressing pressing social and environmental concerns. But how can organizations ascertain whether their isolated or collaborative efforts are truly generating positive impact to their target populations? Measuring real impact requires addressing several issues related to causality, comparability, and cost (given that most methods require customized data collection and analysis). In this article, I briefly discuss alternative methods to assess impact and then suggest novel research avenues to inform the debate on how to measure impact and how impact measurement practices can help organizations blend social and economic goals.


Introduction

In the last decades, management scholars have examined ways through which firms can reconcile financial and social goals (Barnett, 2007Barnett, M. L. (2007). Stakeholder influence capacity and the variability of financial returns to corporate social responsibility. Academy of Management Review, 32(3), 794-816.; Battilana & Dorado, 2010Battilana, J., & Dorado, S. (2010). Building sustainable hybrid organizations: The case of commercial microfinance organizations. Academy of Management Journal, 53(6), 1419-1440.; Margolis, Elfenbein, & Walsh, 2008Margolis, J. D., Elfenbein, H. A., & Walsh, J. P. (2008). Do well by doing good? Don’t count on it. Harvard Business Review, 86(1), 19.). Corporate social responsibility practices (McWilliams & Siegel, 2001McWilliams, A., & Siegel, D. (2001). Corporate social responsibility: A theory of the firm perspective. Academy of Management Review, 26(1), 117-127.), bottom of the pyramid strategies (Prahalad, 2004Prahalad, C. K. (2004). The fortune at the bottom of the pyramid. Pearson: Upper Saddle River.), socially-responsible investing (Barnett & Salomon, 2006Barnett, M. L., & Salomon, R. M. (2006). Beyond dichotomy: The curvilinear relationship between social responsibility and financial performance. Strategic Management Journal, 27(11), 1101-1122.), and the pursuit of shared value more generally (London & Hart, 2004London, T., & Hart, S. L. (2004). Reinventing strategies for emerging markets: Beyond the transnational model. Journal of International Business Studies, 35(5), 350-370.; Porter & Kramer, 2011Porter, M. E., & Kramer, M. R. (2011, January–February). Creating shared value. Harvard Business Review, 62-77.) have all emerged from a common desire to build profitable operations that also address pressing social and environmental issues. This effort is naturally aligned with the objective of policy makers and public managers to guarantee that state or privately sponsored activities effectively generate positive outcomes to the population at large (Bryson, Crosby, & Bloomberg, 2015Bryson, J. M., Crosby, B. C., & Bloomberg, L. (2015). Discerning and assessing public value: Major issues and new directions. In J. M. Bryson, B. C. Crosby, & L. Bloomberg (Eds.), Public value and public administration 1–21. Washington: Georgetown University Press.; Moore, 1995Moore, M. H. (1995). Creating public value: Strategic management in government. Cambridge: Harvard University Press.). In fact, private and public interests are becoming increasingly interdependent as for-profit firms, nonprofits, and governments learn and discover opportunities for mutual collaboration (Mahoney, McGahan, & Pitelis, 2009Mahoney, J. T., McGahan, A. M., & Pitelis, C. N. (2009). The interdependence of private and public interests. Organization Science, 20(6), 1034-1052.).

In this setting, a natural question emerges: How can these actors ascertain whether their isolated or collaborative efforts are truly generating positive impact to their target populations? Increasingly, there is growing concern on how to effectively measure the positive effect of their activities, if any, on relevant social and environmental outcomes (Donaldson, Christie, & Mark, 2015Donaldson, S. I., Christie, C. A., & Mark, M. M. (Eds.). (2015). Credible and actionable evidence: The foundation for rigorous and influential evaluations. Los Angeles: SAGE.). Measuring impact, however, is far from a trivial task. First, there is the issue of causality: managers and policy makers would like to know if possible improvements in outcomes were caused by their own effort or instead other confounding factors (Duflo, Glennerster, & Kremer, 2008Duflo, E., Glennerster, R., & Kremer, M. (2008). Using randomization in development economics research: A toolkit. In T. P. Schultz, & J. Strausss (Eds.), Handbook of development economics (pp. 3895–3957). Amsterdam: Elsevier.). Some view that assessing impact requires counterfactual analysis, that is, what would have happened to the target population if the project were not implemented in the first place (Brest & Born, 2013Brest, B. P., & Born, K. (2013). When can impact investing create real impact? Stanford Social Innovation Review, 22-31.). Second, there is the issue of comparability (Kroeger & Weber, 2014Kroeger, A., & Weber, C. (2014). Developing a conceptual framework for comparing social value creation. Academy of Management Review, 39(4), 513-540.). In traditional strategy research, economic performance is measured using standard indicators, such as return on assets or stock market value, compared to a common norm such as average industry performance (e.g. McGahan, 1999McGahan, A. M. (1999). The performance of US corporations: 1981–1994. The Journal of Industrial Economics, 47(4), 373-398.). But how to contrast the outcomes of projects covering distinct areas such as education health or crime prevention? Third, and no less important, there is the issue of cost. Assessing impact often requires intensive data collection and sophisticated analysis—sometimes much like scientific research-, which creates formidable challenges for financially-constrained governments and entrepreneurs.

The objective of this article is to briefly discuss how the measurement of social impact can and stimulate research on the strategies, challenges, and limitations to measure the impact of addressing socio-environmental needs. In the next section I discuss how the measurement of social impact has evolved and then I turn to some comments on how this trend creates numerous opportunities for novel research in business administration.

Measuring impact

I discussed before that the measurement of impact needs to deal with key issues related to causality, comparability, and measurement cost. How have different measurement techniques tackled the tradeoffs involving these dimensions? At a more fundamental level, we can distinguish between two general approaches to assess impact: standardized and project-specific (Lazzarini, Cabral, Pongeluppe, Ferreira, & Rotondaro, 2014Lazzarini, S. G., Cabral, S., Pongeluppe, L. S., Ferreira, L. C. M., & Rotondaro, A. (2014). The best of both worlds? Impact investors and their role in the financial versus social performance debate. In Paper presented at the Academy of Management Conference. Philadelphia.).

Examples of standardized tools to measure impact include IRIS (Impact Reporting and Investment Standards), GIIRS (Global Impact Investing Report System), and the B Lab Certification. GIIRS uses the “dictionary” of impact variables proposed by IRIS and allows entrepreneurs and investors to self-report their performance on a set of dimensions (governance, workers, community, environment, and focus of the business model). The B Lab Certification is also based on the IRIS/GIIRS system and allows firms to receive an external certification indicating the social orientation of their business model. More recently, entrepreneurs and impact investors have also used the standardized indicators proposed by the United Nations’ Social Development Goals. By definition, standardization allows for an improve comparability of individual assessments of impact. Their open and mostly self-reported nature also tend to reduce measurement costs. However, these standardized tools do not address the issue of causality, that is, whether changes in the target populations of these firms (if any) were caused by the actions that the firms themselves conceived and implemented.

Project-specific approaches to impact assessment try to fill this void by focusing on variables that are more related to firm-level interventions and then adopting procedures to identify the causal effect of the project (Salazar, Husted, & Biehl, 2012Salazar, J., Husted, B. W., & Biehl, M. (2012). Thoughts on the evaluation of corporate social performance through projects. Journal of Business Ethics, 105(2), 175-186.). To address causality, some measurement techniques have followed the so-called principle of additionality (e.g. Brest & Born, 2013Brest, B. P., & Born, K. (2013). When can impact investing create real impact? Stanford Social Innovation Review, 22-31.). In this case, investors assess impact not only considering the performance of the project, but also taking into account its counterfactual, that is, what would have happened without the set of actions engendered by the firm. There are several ways to build counterfactual scenarios using control groups of individuals that were not subject to the intervention (Lazzarini, Pongeluppe, Yoong, & Ito, 2015Lazzarini, S. G., Pongeluppe, L. S., Yoong, P. S. Y., & Ito, N. C. (2015). Guia para a avaliação de impacto socioambiental para utilização em investimento de impacto. TAC (Tecnologias de Administração e Contabilidade), 5(2), 106-118.). A possibility is to compare the outcomes of those affected by the project to a matched group of individuals with similar observable characteristics. Another possibility, which is even more rigorous in terms of the assessment of causality, is to implement randomized controlled trials (RCT). In this method, communities or individuals receiving the benefits of the investment are defined at random, thereby controlling for unobservable factors influencing the outcomes of the intervention (Duflo et al., 2008Duflo, E., Glennerster, R., & Kremer, M. (2008). Using randomization in development economics research: A toolkit. In T. P. Schultz, & J. Strausss (Eds.), Handbook of development economics (pp. 3895–3957). Amsterdam: Elsevier.).

Although project-specific approaches based on verification of additionality allow for greater precision in the assessment of causal effects, they are usually seen as more costly than standardized tools, since they require technical effort to design and analyze the data using more complex econometric techniques and, in some cases, firms will need to collect new data serving as an input to those analyses. In the case of RCTs, their implementation may be even unfeasible, given that in most cases firms themselves choose the individuals or communities they want to target. By focusing on more customized measurements, project-specific approaches also tend to have lower comparability. A way however to increase comparability is to find a common metric to assess outcomes across several types of projects, using for instance techniques to compute the economic return of social interventions (Nicholls, Lawlor, Neitzert, & Goodspeed, 2009Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2009). A guide to social return on investment. Office of the third sector. London: The Cabinet Office.).

Research opportunities

Given this diversity of tools, a natural research question is: What explains the choice of alternative tools to measure impact? For instance, financially-constrained entrepreneurs may find it more difficult to adopt project-specific measures that required customized effort to design measurement plans and collect specialized data. On the other hand, certain types of impact-oriented investors may require their supported organizations to adopt more robust techniques to assess the causality of the intervention. In this sense, researchers could examine the diversity of entrepreneurs and investors and how their distinct orientation and resources explain the adoption of varied tools to assess impact.

If we evolve towards better impact assessment, can it become a new performance dimension in management, complementing financial, market, and productivity-oriented measures? Although the extant literature has tried to examine the potential reconciliation of economic and social dimensions of performance (Barnett & Salomon, 2006Barnett, M. L., & Salomon, R. M. (2006). Beyond dichotomy: The curvilinear relationship between social responsibility and financial performance. Strategic Management Journal, 27(11), 1101-1122.; Margolis et al., 2008Margolis, J. D., Elfenbein, H. A., & Walsh, J. P. (2008). Do well by doing good? Don’t count on it. Harvard Business Review, 86(1), 19.), there is still room to adopt more refined project-specific measures of impact. In strategic management, competitive advantage is usually measured as the long-term performance of the firm compared to the industry norm (McGahan, 1999McGahan, A. M. (1999). The performance of US corporations: 1981–1994. The Journal of Industrial Economics, 47(4), 373-398.). In the future, as researchers collect more data about project-specific outcomes, it may be possible to overcome the comparability limitations of these specific measurements by assessing how certain firm-specific outcomes compare to the outcomes generated by other firms in similar activities (in the same way as financial analysts compare the returns of certain firms to their peers in the same industry).

Measures of impact can also be used to compensate managers and investors for their superior social outcomes (Bugg-Levine, Kogut, & Kulatilaka, 2012Bugg-Levine, A., Kogut, B., & Kulatilaka, N. (2012). A new approach to funding social enterprises. Harvard Business Review, 90(1/2), 118-123.; Social Finance, 2009Social Finance. (2009). Social impact bonds: Rethinking finance for social outcomes. London.). Consider the case of the so-called social impact bonds (SIBs), a contractual mechanism through which private investors provide funding to social projects on a pay-for-impact basis (that is, the government repays investors only if certain targets of impact are met). The first SIB project was executed in the UK in the context of prisons, as an effort to reduce recidivism. The chosen approach to measure impact was project-specific and based on verification of additionality. Recidivism in the prison affected by the investment was compared to a matched control group of prisoners in other localities not affected by the investment. In this case, impact was only achieved—and investors compensated—if recidivism in the focal prison was reduced to greater extent than in the control group of prisoners. In this sense, there is substantial room, in this case, to study how distinct types of measurements can or should be used in those pay-for-impact contracts. For instance, additionality-based assessments have the advantage to more precisely estimate improvements in outcomes caused by new investments. However, they add complexity and even additional risks to investors if comparison groups are not properly designed.

Scholars can also scrutinize how resources, management practices, and organizational forms affect the way firms measure and achieve impact. For instance, there is growing interest in the analysis of heterogeneous management practices and how they affect firm-level profitability and productivity (Bloom & Van Reenen, 2010Bloom, N., & Van Reenen, J. (2010). Why do management practices differ across firms and countries? The Journal of Economic Perspectives, 24(1), 203-224.; Bromiley & Rau, 2014Bromiley, P., & Rau, D. (2014). Towards a practice based view of strategy. Strategic Management Journal, 35(8), 1249-1256.). Arguably, measuring impact can itself be an important management practice that can greatly vary across firms and sectors. Distinct types of practices can also influence the ability of firms to generate impact, and the adoption of these practices can also be a function of firm-specific resource endowments (Teodorovicz, Lazzarini, Cabral, & Nardi, 2017Teodorovicz, T., Lazzarini, S. G., Cabral, S., & Nardi, L. (2017). Integrating organizational, resource-based and practicebased views of heterogeneous performance. In Paper presented at the Academy of Management Conference Atlanta.). Future research should delve into the practice- and resource-based determinants of the tools for impact assessment adopted by firms and the effectiveness of those assessments.

Although not intended to exhaust all possible avenues for future research, these questions and issues illustrate some themes and issues that can be pursued by future work. With more scholarly effort in this field, we can not only better inform managers about the potential and limitations of alternative ways to measure impact, but also expand and refine theories of how firms can generate and combine social and economic value.

  • This short article is based on the proposal for a Professional Development Workshop at the Academy of Management Conference in 2017, jointly organized with Sandro Cabral and Aldo Musacchio.

References

  • Barnett, M. L. (2007). Stakeholder influence capacity and the variability of financial returns to corporate social responsibility. Academy of Management Review, 32(3), 794-816.
  • Barnett, M. L., & Salomon, R. M. (2006). Beyond dichotomy: The curvilinear relationship between social responsibility and financial performance. Strategic Management Journal, 27(11), 1101-1122.
  • Battilana, J., & Dorado, S. (2010). Building sustainable hybrid organizations: The case of commercial microfinance organizations. Academy of Management Journal, 53(6), 1419-1440.
  • Bloom, N., & Van Reenen, J. (2010). Why do management practices differ across firms and countries? The Journal of Economic Perspectives, 24(1), 203-224.
  • Brest, B. P., & Born, K. (2013). When can impact investing create real impact? Stanford Social Innovation Review, 22-31.
  • Bromiley, P., & Rau, D. (2014). Towards a practice based view of strategy. Strategic Management Journal, 35(8), 1249-1256.
  • Bryson, J. M., Crosby, B. C., & Bloomberg, L. (2015). Discerning and assessing public value: Major issues and new directions. In J. M. Bryson, B. C. Crosby, & L. Bloomberg (Eds.), Public value and public administration 1–21 Washington: Georgetown University Press.
  • Bugg-Levine, A., Kogut, B., & Kulatilaka, N. (2012). A new approach to funding social enterprises. Harvard Business Review, 90(1/2), 118-123.
  • Donaldson, S. I., Christie, C. A., & Mark, M. M. (Eds.). (2015). Credible and actionable evidence: The foundation for rigorous and influential evaluations Los Angeles: SAGE.
  • Duflo, E., Glennerster, R., & Kremer, M. (2008). Using randomization in development economics research: A toolkit. In T. P. Schultz, & J. Strausss (Eds.), Handbook of development economics (pp. 3895–3957). Amsterdam: Elsevier.
  • Kroeger, A., & Weber, C. (2014). Developing a conceptual framework for comparing social value creation. Academy of Management Review, 39(4), 513-540.
  • Lazzarini, S. G., Cabral, S., Pongeluppe, L. S., Ferreira, L. C. M., & Rotondaro, A. (2014). The best of both worlds? Impact investors and their role in the financial versus social performance debate. In Paper presented at the Academy of Management Conference. Philadelphia
  • Lazzarini, S. G., Pongeluppe, L. S., Yoong, P. S. Y., & Ito, N. C. (2015). Guia para a avaliação de impacto socioambiental para utilização em investimento de impacto. TAC (Tecnologias de Administração e Contabilidade), 5(2), 106-118.
  • London, T., & Hart, S. L. (2004). Reinventing strategies for emerging markets: Beyond the transnational model. Journal of International Business Studies, 35(5), 350-370.
  • Mahoney, J. T., McGahan, A. M., & Pitelis, C. N. (2009). The interdependence of private and public interests. Organization Science, 20(6), 1034-1052.
  • Margolis, J. D., Elfenbein, H. A., & Walsh, J. P. (2008). Do well by doing good? Don’t count on it. Harvard Business Review, 86(1), 19.
  • McGahan, A. M. (1999). The performance of US corporations: 1981–1994. The Journal of Industrial Economics, 47(4), 373-398.
  • McWilliams, A., & Siegel, D. (2001). Corporate social responsibility: A theory of the firm perspective. Academy of Management Review, 26(1), 117-127.
  • Moore, M. H. (1995). Creating public value: Strategic management in government Cambridge: Harvard University Press.
  • Nicholls, J., Lawlor, E., Neitzert, E., & Goodspeed, T. (2009). A guide to social return on investment. Office of the third sector London: The Cabinet Office.
  • Porter, M. E., & Kramer, M. R. (2011, January–February). Creating shared value. Harvard Business Review, 62-77.
  • Prahalad, C. K. (2004). The fortune at the bottom of the pyramid Pearson: Upper Saddle River.
  • Salazar, J., Husted, B. W., & Biehl, M. (2012). Thoughts on the evaluation of corporate social performance through projects. Journal of Business Ethics, 105(2), 175-186.
  • Social Finance. (2009). Social impact bonds: Rethinking finance for social outcomes London.
  • Teodorovicz, T., Lazzarini, S. G., Cabral, S., & Nardi, L. (2017). Integrating organizational, resource-based and practicebased views of heterogeneous performance. In Paper presented at the Academy of Management Conference Atlanta.

Publication Dates

  • Publication in this collection
    Jan-Mar 2018
Universidade de São Paulo Avenida Professor Luciano Gualberto, 908, sala F184, CEP: 05508-900, São Paulo , SP - Brasil, Telefone: (11) 3818-4002 - São Paulo - SP - Brazil
E-mail: rausp@usp.br