Research impact – what is it, after all? Editorial impact series part 1

Introduction

Nothing in science has any value to society if it is not communicated, and scientists are beginning to learn their social obligations.

Anne Roe (American psychologist 1904 – 1991), The Making of a Scientist (1953), p. 17.

The impact of research on society or societal impact, has been widely discussed at an international level, not only because of its undeniable importance to justify the resources invested in research but also because of its inclusion in the assessments of research institutions to obtain financing. For a long time, the research impact assessment had taken into account only the scientific impact, that is, in essence, the relevance of the published research to other subsequent research. The relevance of the research, in this case, is measured mainly through citations obtained in other academic publications, especially scientific journals, which gave rise to the impact indicators of journals that we currently use (e.g. JCR, SJR, Index H, etc.).

Despite not being new, this discussion on the impact of science on society has gained more momentum in the past two decades due to some reasons:

• a demand for transparency of investments in research financed with public resources, mainly in terms of the generated outcomes;

• a more significant concern with the use of scientific evidence to support decision-making (in both public and business sectors; and

• the need to demonstrate how research is contributing to the society’s grand challenges – in other words, its relevance and impact on society – and not just for science itself (Brook, 2018; Alla, Hall, Whiteford, Head, & Meurk, 2017).

Regarding specifically in the management field, the debate about “research rigour vs relevance” to organisational practice also has a long history (Vermeulen, 2005). With the development of management as a science, it is conveyed that there has been a separation and a distance between administrative science and professional practice in the area, creating a gap between what is produced in academia and what is done in organisations. As a result, it is stated that much of the research carried out at universities has little use or relevance for business and practitioners’ needs (Banks et al., 2016; Hughes, Webber, & O'Regan, 2019).

In this sense, we would like to contribute to a more informed and critical discussion on this topic. In this editorial, we aim to present the (several) definitions of impact. In other forthcoming issues of RAUSP Management Journal, we will deepen this theme by addressing the forms of impact assessment, the criticisms on the research impact, improvements and contributions in the academic context.

What is research impact?

The literature presents several different definitions for the impact of research on society. This variety reflects the different ways of thinking about the nature of research and its either potential or actual contribution. This type of impact is perceived differently by institutions, funding agencies, countries, within different research contexts and cultures. Consequently, this conceptual diversity generates different approaches to demonstrate and assess this type of impact (Pedersen, Grønvad, & Hvidtfeldt, 2020). Table 1 presents some of these definitions.

We can notice that in all these definitions the impact is treated as some kind of contribution, influence, value, change, effect or benefit that scientific research brings to society in its most diverse spheres (economic, social, health, public policies, etc.), directly or indirectly.

Table 1.
Different definitions for research impact

Definition	Author(s)
Demonstrable contribution that excellent research makes to society and the economy, of benefit to individuals, organisations and nations	Edwards and Meagher (2020)
Effects and outcomes of being involved in conducting research, and the value and benefit associated with using knowledge produced by researchers	Peter, Kothari, and Masood (2017)
Direct or indirect contribution of research processes or outputs that have informed (or resulted in) the development of new policy/practices or revisions of existing policy/practices, at various levels of governance	Alla et al. (2017)
Changes in awareness, knowledge and understanding, ideas, attitudes and perceptions and policy and practice as a result of research	Morton (2015)
Contribution of research to knowledge production, capacity-building, informing policy or product development, health and health sector benefits and broader social and economic benefits	Donovan (2011)
Effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia	Research Excellence Framework (REF) (2020)
Effect or benefit perceived by society, derived from products developed in the scope of postgraduate activities	Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) (2019)

Types of impact

Besides these different views and definitions of research impact, a common distinction found in the literature is between instrumental or conceptual impact. The instrumental type refers to the direct impact of research on actions, problem-solving and decision-making by policymakers and practitioners. It is the influence of some specific research in making a decision or in defining the solution to a particular problem. The conceptual impact occurs when research influences the knowledge, understanding and attitudes of policymakers and practitioners, changing the ways of thinking on a specific question or choice or playing an awareness role (Nutley, Walter, & Davies, 2007).

Additionally, there have been several studies addressing different perspectives on who or what can be the object of the eventual impact of research. Some of these perspectives are:

• Economic impact: most studies on the impact of science on society focus on economic impacts, for which several measurement econometric models have been created since the 1950s (Bornmann, 2013; Godin & Doré, 2005). The economic impact is associated with the benefit that the research brings to the economic capital of a nation, region or even an organisation (Bornmann, 2013). It is usually associated to efficiency or productivity gains, generation of competitive advantage or jobs created, seeking to associate the investments in research with the generated returns (return on investment - ROI) (Gunn & Mintrom, 2017; Russell Group Universities, 2012; Bessette, 2003). Activities associated with this type of impact include the generation of intellectual property (patents deposited and granted) and its commercialisation (e.g. funds received through licensing), new business generated at universities (spin-offs), etc (Perkman et al., 2015; Kalika, Shenton, & Dubois, 2016; Bornmann, 2013; Yeo, 2018).

• Social impact: refers to the contribution of research and the generated knowledge to changes in behaviours, customs, habits and practices (among other aspects) that generates well-being and quality of life for individuals and groups (Godin & Doré, 2005). Some researchers, due to the confusion between societal impact and social impact, seek to associate social impact with the set of other impacts listed below (Bornmann, 2013). In this sense, societal impact would then be the umbrella term for impact of research on society, and social impact a more specific term representing the impact of research on the various social aspects of life.

• Impact on public policies: it is related to how scientific knowledge informs and influences the process that involves public policies and responsible managers. It includes changes in interests, attitudes and knowledge concerning an issue of public interest that involves science and technology, contribution to public debate and policy negotiation, as well as the influence of research in decision-making and in public actions that generate changes in a specific policy, legislation or project in the policy community (Godin & Doré, 2005; Escribano-Ferrer, Webster, & Gyapong, 2017).

• Organisational impact: it is the impact of research on the activities of organisations, contributing to their management, including planning, work organisation technologies, general management activities (marketing, accounting, purchasing, etc.) and human resources (Godin & Doré, 2005). This type of impact includes, for example, contributions to improving the organisational performance due to changes in practices and processes, the understanding the importance of diversity (gender, ethnicity, etc.) leading to changes in the organisation’s internal policies, etc.

• Cultural impact: these are benefits to cultural capital in all its forms. When research contributes by enriching and stimulating the creativity, imagination and cultural production of individuals and groups, in addition to collaborating in the understanding and preservation of the national cultural heritage (Brook, 2018; Bornmann, 2013; Russell Group Universities, 2012). Examples include collaboration with museum professionals resulting in improvements for the preservation of cultural heritage; new ways of thinking that influence creative practice; involvement with marginalised audiences leading to an increase in cultural participation (Research Excellence Framework [REF], 2019).

• Educational impact: it is related to contributions of research to the functioning of education systems at its various levels, including curricula, pedagogical tools, instruments, processes and technologies that contribute both to the improvement of teaching and the learning of new knowledge (Godin & Doré, 2005; Coordenação de Aperfeiçoamento de Pessoal de Nível Superior [CAPES], 2019). Examples include influence on the design and ways of delivering subjects to educational institutions; changes in the curriculum proposed by research; contribution to the development of innovative pedagogical proposals (Research Excellence Framework [REF], 2019).

In addition to these types of impacts above, we can also find some additional categories in the literature, such as environmental, health, technological, legal and capacity-building impacts. As the reader can notice, many or maybe all of the impact types listed here can be achieved or addressed somehow in management research. We will go back to this subject in some of the next editorials.

References

• Alla, K., Hall, W. D., Whiteford, H. A., Head, B. W., & Meurk, C. S. (2017). How do we define the policy impact of public health research? a systematic review. Health Research Policy and Systems, 15(1), 84 10.1186/s12961-017-0247-z. 10.1186/s12961-017-0247-z.
  » https://doi.org/10.1186/s12961-017-0247-z
• Banks, G. C., Pollack, J. M., Bochantin, J. E., Kirkman, B. L., Whelpley, C. E., & Boyle, E. H. (2016). Management's science-practice gap: a grand challenge for all stakeholders. Academy of Management Journal, 59(6), 2205–2231. 10.5465/amj.2015.0728.
  » https://doi.org/10.5465/amj.2015.0728
• Bessette, R. W. (2003). Measuring the economic impact of university-based research. The Journal of Technology Transfer, 28(3/4), 355–361. 10.1023/A:1024917601088.
  » https://doi.org/10.1023/A:1024917601088
• Bornmann, L. (2013). What is societal impact of research and how can it be assessed? a literature survey. Journal of the American Society for Information Science and Technology, 64(2), 217–233. 10.1002/asi.22803.
  » https://doi.org/10.1002/asi.22803
• Brook, L. (2018). Evidencing impact from art research: analysis of impact case studies from the REF 2014. The Journal of Arts Management, Law, and Society, 48(1), 57–69. 10.1080/10632921.2017.1386148.
  » https://doi.org/10.1080/10632921.2017.1386148
• Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES). (2019). GT impacto e relevância econômica e social: Relatório Final de Atividades, Brasília, DF: CAPES.
• Donovan, C. (2011). State of the art in assessing research impact: introduction to a special issue. Research Evaluation, 20(3), 175–179. 10.3152/095820211X13118583635918.
  » https://doi.org/10.3152/095820211X13118583635918
• Edwards, D. M., & Meagher, L. R. (2020). A framework to evaluate the impacts of research on policy and practice: a forestry pilot study. Forest Policy and Economics, 114, 101975, doi: 10.1016/j.forpol.2019.101975.
  » https://doi.org/10.1016/j.forpol.2019.101975
• Escribano-Ferrer, B., Webster, J., & Gyapong, M. (2017). Assessing the impact of health research on health policies: a study of the dodowa health research Centre, Ghana. BMC Health Services Research, 17(1), 435. 10.1186/s12913-017-2383-0.
  » https://doi.org/10.1186/s12913-017-2383-0
• Godin, B., & Doré, C. (2005). Measuring the impacts of science: beyond the economic dimension., Paper presented at the HIST Lecture, Helsinki Institute for Science and Technology Studies, Helsinki, available at: http://www.csiic.ca/PDF/Godin_Dore_Impacts.pdf
  » www.csiic.ca/PDF/Godin_Dore_Impacts.pdf
• Gunn, A., & Mintrom, M. (2017). Evaluating the non-academic impact of academic research: design considerations. Journal of Higher Education Policy and Management, 39(1), 20–30. 10.1080/1360080X.2016.1254429.
  » https://doi.org/10.1080/1360080X.2016.1254429
• Hughes, T., Webber, D., & O'Regan, N. (2019). Achieving wider impact in business and management: analysing the case studies from REF 2014. Studies in Higher Education, 44(4), 628–642. 10.1080/03075079.2017.1393059.
  » https://doi.org/10.1080/03075079.2017.1393059
• Kalika, M., Shenton, G., & Dubois, P. (2016). What happens if a business school disappears? the intellectual foundations of BSIS. Journal of Management Development, 35(7), 878–888. 10.1108/JMD-10-2014-0120.
  » https://doi.org/10.1108/JMD-10-2014-0120
• Morton, S. (2015). Progressing research impact assessment: a 'contributions' approach. Research Evaluation, 24(4), 405–419. 10.1093/reseval/rvv016.
  » https://doi.org/10.1093/reseval/rvv016
• Nutley, S. M., Walter, I., & Davies, H. T. O. (2007). Using Evidence: How Research Can Inform Public Services, Bristol, UK: Policy Press.
• Pedersen, D. B., Grønvad, J. F., & Hvidtfeldt, R. (2020). Methods for mapping the impact of social sciences and humanities — a literature review. Research Evaluation, 29(1), 4–21. 10.1093/reseval/rvz033.
  » https://doi.org/10.1093/reseval/rvz033
• Perkman, M., Fini, R., Ross, J.-M., Salter, A., Silvestri, C., & Tartari, V. (2015). Accounting for universities' impact: using augmented data to measure academic engagement and commercialisation by academic scientists. Research Evaluation, 24(4), 380–391. 10.1093/reseval/rvv020.
  » https://doi.org/10.1093/reseval/rvv020
• Peter, N., Kothari, A., & Masood, S. (2017). Identifying and understanding research impact: a review for occupational scientists. Journal of Occupational Science, 24(3). 10.1080/14427591.2016.1277547.
  » https://doi.org/10.1080/14427591.2016.1277547
• Research Excellence Framework (REF). (2019). REF 2021: Panel criteria and working methods. Retrieved from http://www.ref.ac.uk/publications/panel-criteria-and-working-methods-201902/
  » www.ref.ac.uk/publications/panel-criteria-and-working-methods-201902/
• Research Excellence Framework (REF). (2020). REF impact. Retrieved from https://re.ukri.org/research/ref-impact/
  » https://re.ukri.org/research/ref-impact/
• Russell Group Universities. (2012). The Social Impact of Research Conducted in Russell Group Universities, London: Russell Group Universities.
• Vermeulen, F. (2005). On rigor and relevance: fostering dialectic progress in management research. Academy of Management Journal, 48(6). 10.5465/amj.2005.19573102.
  » https://doi.org/10.5465/amj.2005.19573102
• Yeo, B. (2018). Societal impact of university innovation. Management Research Review, 41(11), 1309–1335. 10.1108/MRR-12-2017-0430.
  » https://doi.org/10.1108/MRR-12-2017-0430

Publication Dates

• Publication in this collection
  05 Oct 2020
• Date of issue
  Jul-Sep 2020