Academic research for impact

1. INTRODUCTION

Relevant research is of great interest. I have commented on this topic over the years. For example, I spoke recently on the importance of research relevance in the American Institute of Certified Public Accountants/Chartered Institute of Management Accountants (AICPA/CIMA) Academic Research for Impact Webinar Series (To view the recorded webinar, please see AICPA/CIMA Academic Research for Impact Webinar Series, Part 1 - https://sway.office.com/kbtAGFpCsoDn9bRT?ref=Link). I have also spoken about it at several doctoral seminars around the world; written about in the European Accounting Association (EAA) Newsletter (Van der Stede, 2012); and debated related challenges on an AICPA panel of the American Accounting Association (AAA)’s 2019 Management Accounting Section meeting (Make Management Accounting Research Happen: Opportunities, Challenges and Evidence from Attempts to Achieve Research Impact, Fort Lauderdale, 4 January 2019).

The issue of research relevance is multifaceted and can be debated from numerous angles. In this editorial, I will take one particular angle - that is, by thinking about impact more explicitly in terms of how our research can seek to better connect with practice and the practitioner. Thus, I will not reflect on what research impact or relevance means or implies for a researcher’s career or how it is rated, “measured”, or debated in the academic community, including in universities and by the scholarly journals. Instead, I am taking a “practice view” on impact rather than an institutional view (e.g., what it means in terms of academic careers, university funding, etc.) or an epistemological view (e.g., what it means in terms of the nature of knowledge, publishing etc.). As a further general disclaimer, this piece should be read in the context of the “administrative sciences” (or management) as an applied science to the social sciences. That is broad enough, but obviously not all encompassing, and thus, readers should be cautioned about potentially misguided generalizations to other sciences.

2. WHAT IS IMPACT, OR RATHER, “RESEARCH RELEVANCE”?

With such a practice view in mind, here is my favorite definition of “impact”-or rather, “relevance” - quoting from Toffel (2016, p. 1493): “Relevant research papers [are] those whose research questions address problems found (or potentially found) in practice and whose hypotheses connect independent variables within the control of practitioners to outcomes they care about using logic they view as feasible.” I explain in both the AICPA/CIMA Webinar and EAA Newsletter (Van der Stede, 2012) why I prefer to focus on relevance instead of impact: because relevance is more directly under the researcher’s control and probably a reasonable (though not guaranteed) precursor for eventual impact. Or, put bluntly, that someone fails to have eventual impact by some measure is probably uncontrollable to some extent, but to do research that is not relevant is less excusable. That is not to say that establishing relevance for any research is easy and/or not subject to tastes and biases (see also Van der Stede [2012]). That said, I feel strongly that a researcher must be interested in the relevance of their research from the get-go; it is something the researcher must think about very carefully from the start. This was also my main message in the AICPA/CIMA Webinar and during the other occasions where I spoke on this topic.

I adopt a broad view of what such relevant research could be, and I have no intention to be even remotely prescriptive about it. With an imponderable number of issues that managers face in their organizations, it is simply inconceivable that there would be a finite set of interesting questions for researchers to address that are potentially relevant for the practitioners in that chosen field. I am also not opining on how the research should be done (by which method) or through which disciplinary lens (using whichever theory). Method and theory should be irrelevant for the question of research relevance; instead, its defining characteristic is whether the research has contributed knowledge (in whichever academically rigorous way) to a “problem found (or potentially found) in practice” (Toffel, 2016). Thus, my ultimate focus is on the issue of how researchers can make sure to address problems that practitioners face in practice by way of research that delineates the means through which the practitioners can affect outcomes they care about.

3. THE RESEARCH RELEVANCE TRICHOTOMY

There are three key parts to relevance expressed in the prior section. First, there is a relevant problem - that is, the research addresses a problem that practitioners face. Second, the research should involve independent variables within the control of practitioners. These are the dials that the practitioner could conceive turning. Third, there must be an effect.

The “desired effect” of course could also be called the “solution” to the problem - but I try to stop short of being too normative or creating unreasonable expectations that anything short of “the” solution would amount to having conducted “irrelevant” research. Not in the least. Indeed, as I explain in the webinar and in my earlier pieces on this topic, research is an inevitably long, drawn-out process like peeling an onion layer by layer; a process that may ironically not have had “impact” or a clear “solution” as its explicit or even achievable objective, but rather should be focused on a gradual, well-executed, multi-faceted understanding of a justifiably relevant problem where each piece of the research, however, should nonetheless not lose sight of the trichotomy elaborated in this article by having a relevant problem, including controllable variable(s), and aiming to achieve some feasible outcome(s). This entails some plausible remedies to the problem or part problem. Simplified, there is thus a relevant “problem” to which the research offers some “solution” by having studied “variables” that connect the two and which the practitioner can influence, work with, act on, or do something about.

This is the core trichotomy at the heart of this editorial: [relevant] problem - [controllable] variable(s) - [feasible] outcome, where relevant, controllable, and feasible all must be seen from the practitioner’s perspective. This is similar in spirit to another definition I quote (Kieser et al., 2015, p. 144): “Broadly speaking, research results can be said to be practically relevant if they influence management practice; that is, if they lead to the change, modification, or confirmation of how managers think, talk, or act.” I hasten to add that another nice feature of this “definition” is that “impact” is not about “changing the world” - it inevitably always is much more modest. But it should not be vacuous either: the research must engage with some relevant problem, suggest an outcome, and cover some way(s) that practitioners could bring about that outcome.

How to then try and make sure that any given piece of research has these three key elements covered? Again, modesty is fine, even desired. But the researcher’s focus on and awareness about these three bits and the determination to critically challenge oneself that the three bits are present in the research should be undiminished and uncompromising. And this should be the case from the start and throughout the research project; not as an afterthought or an ex-post justification.

4. AN ILLUSTRATION

Let me try and illustrate this with a recent research project of myself with a team of co-authors (Avagyan et al., in press). This is an illustration from the management accounting subarea of accounting, though the research is really on the interface of accounting and marketing (and was published in a marketing journal). There are countless other illustrations that could be used from any other subarea of accounting. In fact, most published research should be usable as an illustration because relevance matters to journal editors as well (see also Van der Stede [2012]). Surely one common reason why papers get rejected is due to a lack of relevance. Thus, relevance is almost assuredly a necessary condition also for publication in an academic journal. That said, relevance is obviously not a sufficient condition for publication, because even if relevant, the research evidently also must be properly theorized and executed.

The “problem” in Avagyan et al. (in press) is whether pitches that include scenarios - that is, a range of outcomes from the best to the worst case - affect the likelihood that an innovation project will get selected? This is an important question given how crucial innovation project selection decisions are to a firm’s competitiveness. There is a fine line to be navigated here. On the one hand, a firm may be (too) prudent, selecting (too many) core innovation projects that target existing markets with products like their current offerings (and too few transformational projects that target new markets with new products). On the other hand, the firm could also be (too) rash, taking (too much) risk by selecting (too many) transformational projects that target new markets with new products (and neglecting core innovation, threatening the firm’s leadership position in the market). Though hugely oversimplified as a juxtaposition, Kodak and Apple are commonly cited to illustrate this tension. That said, most firms probably inherently lean towards core innovations and would, under the right conditions, like to select more transformational innovations in their innovation pipeline. A key question, then, is: does the way in which innovation projects are pitched influence this?

This “problem” sounds at least interesting, but how do we really know it is relevant - a problem that keeps practitioners awake at night? In our team of researchers, the senior co-authors had first-hand experience that this was an important concern through direct contact: some consulting-related, some executive education. Some researchers may balk at that and/or not have such opportunities for consulting or executive education even if they wanted to. That is fine, and even very real, for junior academics for example. Note, however, that I said that direct contact was one way for us to have a finger on the pulse; I did not say it is the only one.

Indeed, we also did a thorough scanning of the literature, including, importantly, of the practitioner literature. This is something that everyone should, and can, do! For example, in a survey of the literature (Graham & Harvey, 2001), we found that 48.5% of 392 surveyed firms always or almost always use scenario analysis, and 51.5% never or infrequently do. This is interesting because about half for some reason do use it, and the other half do not. Hence, there is a good prevalence of use, but it is not total (which would make it less relevant to study). Surely, there must be good reasons for the one-half to find it useful, but equally good reasons for the other half not to. What are those reasons? Why is this? What conditions might this be determined by? These questions are all telltales that you are on to something potentially relevant.

Another practitioner source - Bain’s Management Tools & Trends survey of over a thousand managers (Bain & Company, 2017) - suggested to us that about one-fifth of the respondents use scenarios. Again, this is a “good number” though also lower than the overall mean of 30% use across the 25 popular management tools included in the survey. Thus, same reflections as above: there seems to be enough firms that use scenarios presumably because they must be useful, but there must also be something about scenarios that causes other firms to not find them useful. Why is that? What is it?

We also spoke with quite a few executives as researchers, interviewing them. Do not be shy to reach out to practitioners. Many are remarkable willing to talk about their businesses, including the problems they face. The same picture emerged from these interviews. For example, one executive said: “If you put in front of people a positive and negative case, they will only see the negative. They will not see the positive.” If that is the case (which may be an interesting angle that our research could pursue), then presenting scenarios where the worst case is particularly negative may be akin “shooting yourself in the foot”. Might this be a reason why use of scenarios is patchy? Could this be particularly an unintended consequence for inherently more risky projects whose risk may be perceptually exacerbated when using scenario presentations?

Armed with this, we knew there was an important, interesting problem to be researched. In Toffel’s words: we knew we had a “problem found (or potentially found) in practice” on our hands. Then came the more academic parts of examining the prior literature and designing our study. In doing so, however, we kept revisiting the “problem we found in practice” because we needed, again in Toffel’s words, to make sure that our research included “independent variables within the control of practitioners” to produce “outcomes they care about using logic they view as feasible” (Toffel, 2016, p. 1493).

In a nutshell and cutting out various other results that can be gleaned from Avagyan et al. (in press), our study shows that, across the board (i) scenario presentation dominates no scenarios, but (ii) small-range scenarios dominate large-range ones. Thus, one implication of our study is that firms should help project teams present small-range rather than large-range scenarios. The reason is that, as wide(r) scenarios are presented, not only may the project team be perceived as not having done its homework (i.e., the project team is perceived as less expert), it may also trigger the well-known negativity bias. That is, decision-makers are susceptible to place more weight on negative information than on positive information, thereby increasing their perception of the project’s risk. It is a small step then from decision-makers’ perception that the project team is less expert and/or its project riskier to consequently find that the project is more likely to be rejected.

5. SO WHAT?

Why is this result an “outcome the(se) practitioners care about”? Well, this result is relevant for the half of our own surveyed executives (which corresponds roughly with the half we found from other sources) that currently do not require scenario presentation. It is also relevant for the 79% of surveyed firms that would like to select more transformational than core innovation projects, but who either use no scenarios or, when they do, often use large-range ones. Both no scenarios and large-range scenarios hinder the selection of potentially worthwhile (transformational) innovation projects. That is why they care.

But given that this is a relevant issue about which practitioners care to get it right, what can our research suggest doing about it? We have an “independent variable within the control of practitioners” (Toffel, 2016) because it is their choice whether to use, or not use, scenarios. But can managers influence the range of the scenarios that their project teams tend to generate? That is more difficult, but not impossible. Indeed, although the implication to present small-range scenarios may seem apparent, it is not evident in a sizable number of firms where we observe that project teams present scenarios that are too wide (57% in one of our studies). Our interviewees highlighted several reasons why project teams present large-range scenarios. For instance, some project teams may not be “focused enough” or may not do “strong pre-work to validate the assumptions of their project.” Project teams may also fail to collect sufficient information on potential customers, possible competitive actions, and major costs of maturing their project, which are especially uncertain for transformational innovations, and thus, pose quite a challenge. What then can firms do? Practically, firms can take specific actions to encourage small-range scenario presentation, minimally by way of setting expectations, but additionally also by providing resources and training (see Avagyan et al. [in press]).

6. CONCLUSION

This example hopefully illustrates the importance of keeping the relevant problem - controllable variable(s) - feasible outcome(s) trichotomy in mind from the start, and throughout, your research. You must keep asking yourself, and answering, the question: is my research relevant? Ignoring it or not sufficiently challenging yourself as a researcher on this question is to your own detriment, both for your publication chances (yes, even your academic publication chances) and the gratification you will get from having informed some “problem found in practice” in whatever small though nontrivial way.

Then there is of course still the challenge of getting your academic research out of the strictly academic sphere. You do that indirectly and over time by integrating it into your teaching. You can also do that more directly and more immediately by writing a blog or writing a practice version of your research for the business press (to illustrate this for my own research, see https://blogs.lse.ac.uk/businessreview/?s=Wim+A+Van+der+Stede, for example). You will find that, too, much easier if throughout your research you have kept actively challenging yourself on making sure of the relevance of your work. Hence, this is a matter of translation.

But if the research fails the relevance challenge, then the issue will not be that it is lost in translation - an issue that has been commented on in terms of how best to “bridge” academia and practice and how to make academic research more “accessible” to practitioners -, instead, it is likely to already have been lost before translation. That is something that academics, even if only for their own self-fulfillment, will want to avoid. Working on problems found (or potentially found) in practice is far more intrinsically rewarding. And, after the research is completed and published, relevance is also what keeps your research paying off.

References

• Avagyan, V., Camacho, N., Van der Stede, W. A., & Stremersch, S. (in press). Financial projections in innovation selection: The role of scenario presentation, expertise, and risk. International Journal of Research in Marketing.https://doi.org/10.1016/j.ijresmar.2021.10.009
  » https://doi.org/10.1016/j.ijresmar.2021.10.009
• Bain & Company (2017). Management tools & trends https://www.bain.com/insights/management-tools-and-trends-2017
  » https://www.bain.com/insights/management-tools-and-trends-2017
• Graham, J. R., & Harvey, C. R. (2001). The theory and practice of corporate finance: Evidence from the field. Journal of Financial Economics, 60(2-3), 187-243. https://doi.org/10.1016/S0304-405X(01)00044-7
  » https://doi.org/10.1016/S0304-405X(01)00044-7
• Kieser, A., Nicolai, A. & Seidl, D. (2015). The practical relevance of management research: Turning the debate on relevance into a rigorous scientific research program. Academy of Management Annals, 9(1), 143-233. https://doi.org/10.5465/19416520.2015.1011853
  » https://doi.org/10.5465/19416520.2015.1011853
• Toffel, M. W. (2016). Enhancing the practical relevance of research. Production and Operations Management, 25(9), 1493-1505. https://doi.org/10.1111/poms.12558
  » https://doi.org/10.1111/poms.12558
• Van der Stede, W. A. (2012). Research impact and relevance. EAA Newsletter, 39(3), 20-21. http://www.eaa-online.org/userfiles/file/EAA-Newsletter-Nr39-2012(3).pdf
  » http://www.eaa-online.org/userfiles/file/EAA-Newsletter-Nr39-2012(3).pdf

Publication Dates

• Publication in this collection
  29 Apr 2022
• Date of issue
  May-Aug 2022