Policies for knowledge generation in the catch-up process: evaluation and lessons

Políticas de geração de conhecimento no processo de catch-up: avaliação e lições

Paulo Henrique Assis Feitosa About the author

Abstract

Public intervention to support knowledge generation is recognized as a fundamental strategy that enables the closing the gap between latecomers and forerunners economies. Evaluations and academic studies have been conducted on a wide variety of interventions and rapidly expanded what is known about these policies instruments. However, much of the evidence report conflicting results and can lead to misunderstandings about the potential of these interventions. This article offers a systematic review and a critical discussion of what the literature has to say about the effectiveness of these instruments. It is argued that learning about the effectiveness of interventions requires a capacity to seize and interpret its effects. Specifically, two critical factors are emphasized to understand the potential of these policies, which are their sequence of implementation and the context-dependence of interventions. Since none of these issues has been investigated in-depth, some aspects are discussed to guide future evaluations.

Keywords:
Catch-up; Evaluation; Policy instruments; Knowledge

Resumo

A intervenção pública de apoio à geração de conhecimento é reconhecida como uma estratégia fundamental que permite reduzir a distância entre as economias retardatárias e precursoras. Avaliações e estudos acadêmicos foram realizados em uma ampla variedade de intervenções e expandiram o que sabemos sobre esses instrumentos de políticas. No entanto, muitas das evidências relatam resultados conflitantes e podem levar a mal-entendidos sobre o potencial dessas intervenções. Este artigo oferece uma revisão sistemática e uma discussão crítica do que a literatura tem a dizer sobre a eficácia desses instrumentos. Argumenta-se que aprender sobre a eficácia das intervenções requer a capacidade de perceber e interpretar seus efeitos. Especificamente, enfatiza dois fatores críticos para compreender o potencial dessas políticas, que são sua sequência de execução e a dependência do contexto das intervenções. Como nenhuma dessas questões foi investigada em profundidade, alguns aspectos são discutidos para orientar as avaliações futuras.

Palavras-chave:
Catch-up; Avaliação; Instrumentos de política; Conhecimento

1. Introduction

The production of knowledge that enables eliminating gaps between latecomers and forerunners countries is essential to the catch-up process (ABRAMOVITZ, 1986ABRAMOVITZ, M. Catching Up, Forging Ahead, and Falling Behind. The Journal of Economic History, v. 46, n. 2, p. 385-406, 1986.). These attempts are undertaken in a world characterized by cumulative polarization between rich and poor economies, in terms of innovative intensity (CASTELLACCI, 2011CASTELLACCI, F. Closing the Technology Gap? Review of Development Economics, v. 15, n. 1, p. 180-197, 2011.) and persistent heterogeneity in Research and Development (R&D) intensities among firms in the same sector (COAD, 2019COAD, A. Persistent heterogeneity of R&D intensities within sectors: Evidence and policy implications. Research Policy, v. 48, n. 1, p. 37-50, 2019.). Furthermore, in contexts of increasing returns and technological asymmetries, the absence of public policies is enough to result in the intensification of this process of divergence (CIMOLI; PEREIMA; PORCILE, 2019CIMOLI, M.; PEREIMA, J.B.; PORCILE, G. A technology gap interpretation of growth paths in Asia and Latin America. Research Policy, v. 48, n. 1, p. 125-136, 2019.).

The catch-up theoretical framework argues that policies focused on correcting market failures are insufficient to generate the level of knowledge required to free latecomers from the middle-income trap, given the existence of further failures in the catch-up process (ABRAMOVITZ, 1986ABRAMOVITZ, M. Catching Up, Forging Ahead, and Falling Behind. The Journal of Economic History, v. 46, n. 2, p. 385-406, 1986.). The approach proposed by Lee (2019LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.) addresses three main flaws, namely: the “capability failure”, due to the intrinsic difficulty of building innovation capabilities; the “system failure”, which results from missing or weak connections among actors; and the “size failure”, caused by the lack of world-class businesses.

Once the existence of these failures is recognized, evaluations need to inform what works in policymaking and be more efficient and effective in designing future instruments.1 1 This article uses “evaluation” in a broad sense, including commissioned studies to conclude on the effectiveness of policies and academic investigations on intervention-specific issues. Nevertheless, the multidimensional nature of the catch-up phenomenon is a notorious challenge in the evaluation of these instruments. Three factors lead to the need to undertake the evaluations. First, there are several possible outcomes, and the effects are strongly context-dependent, which means that the impacts tend to appear only under certain conditions. Second, the relevant criteria used in assessing the effectiveness of these policies have changed over time and also the locations where they are applied.2 2 Korea is an illustrative example shifting its emphasis from tariffs to R&D subsidies, in the late 1980s (LEE, 2019). Third, the complex combination of policy instruments necessary to deal with the various dimensions of capabilities of firms and industries. The reasoning of this combination depends on the targets to be achieved and vary according to the stage of income status in the transition.

Although recent evaluations have expanded what is known about policy instruments, systematic attempts to take advantage of these advances are remarkably scarce. Moreover, much of the evidence report conflicting results and can lead to misunderstandings about the potential of these interventions. This article fills this gap by offering a review of what the literature has to say about the effectiveness of policies for knowledge generation and critically discusses the potential of evaluating these instruments in the catch-up process. Therefore, the approach is motived by Lee (2019LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.) more as an analytical framework for evaluating these interventions than a normative framework. The article does not provide an exhaustive review of the literature which is significantly wide and comprehensive. Rather, it focuses on recent literature sourced mostly from leading journals.

The paper is structured as follows: Section 2 describes the policies instruments and review of the empirical literature. Section 3 presents the key findings of the review of the existing empirical literature. Some remaining questions and challenges for future research are examined in Section 4. Section 5 summarises the conclusions and discusses the implications.

2. Scope of policies, rationale, and evidence

According to traditional economic rationale, the need for policies to promote knowledge production is due to the existence of market failures. In these situations, given the character of public good of the knowledge, firms are prohibited from fully appropriating the returns to innovation and tend to underinvest in R&D, when compared to a socially optimal level.

In parallel with this stem of research, the catch-up framework has made significant advances in a conceptual framework that addresses additional obstacles that arise when latecomers attempt closing the knowledge gap with forerunners (LEE, 2016LEE, K. Economic catch-up and technological leapfrogging: The path to development and macroeconomic stability in Korea. Cheltenham: Edward Elgar Publishing Ltd., 2016., 2019). These advances represent a portion of an emerging body of work committed to finding “binding constraints” of countries, according to their income levels and structural differences (RODRIK, 2006RODRIK, D. Goodbye Washington Consensus, Hello Washington Confusion? A review of the World Bank’s Economic Growth in the 1990s: Learning from a Decade of Reform. Journal of Economic Literature, v. 44, n. 4, p. 973-987, 2006.) in contrast to attempts of the mainstream to find a universal factor for economic growth.

This article addresses three failures in the knowledge generation in latecomers that radically differ from this conventional view (LEE, 2019LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.). In this context, policies for knowledge generation are justified by the presence of capability failures that hinder the enhancement of the innovation capabilities, the system failure caused by the lack of collaboration between members of an innovation system and the size failure that results from the lack of large firms in the generation, market introduction and diffusion of innovation. Despite the wide variety of classification of policy interventions, this section reviews the instruments according to these failures and organises the set of studies into logical subsets, through a classification of innovations instruments proposed by Edler et al. (2016aEDLER, J. et al. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016a.).3 3 For the sake of objectivity, many auxiliary interventions are absent, for example, the demand-oriented policies.

2.1 Policies for capability failure

The incentives for private R&D are based on the rationale that activities related to R&D generate knowledge spillovers, a critical phenomenon for latecomers countries, where firms have a low R&D capability. Accordingly, the incentives are required due to the chance of underinvestment in R&D, which is caused by the scarcity of capital, the predominance of imperfectly competitive industries and obstacles to the diffusion of knowledge (LEE, 2019LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.). In these economies, private efforts in R&D are required not only for the further absorptive capacity of advanced technologies, but also for building capacities.

There are two main instruments to encourage private investment in R&D. The tax incentives that comprise indirect mechanisms to support R&D and the direct government funding for private R&D, through grants and contracts. The former is a more market-oriented approach because the firm chooses the moment and the level of investment.

The provision of tax incentives to stimulate private R&D has become an increasingly popular policy in the past decade.4 4 Among OECD members, R&D tax incentives is adopted by more than 20 countries. Developed countries have a long tradition of operating these instruments, but they are also widely adopted in emerging countries such as Brazil, Russia, India, China, and South Africa. Given that building innovation capabilities through R&D involves uncertainty of results and asymmetries of information, financial institutions tend to avoid committing operations to this type of effort. Accessing tax incentives changes the incentive structure faced by firms, by lowering the costs of private R&D (tax burden or other types of mandatory contributions by law). They are delivered only after the R&D activity has been undertaken (LARÉDO; KÖHLER; RAMMER, 2016LARÉDO, P.; KÖHLER, C.; RAMMER, C. The impact of fiscal incentives for R&D. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 18-52.).

Regarding the effectiveness of R&D tax incentives, there is a large amount of evidence that tax credits have a significant positive effect on R&D expenditure, both short and long term.5 5 Similar conclusions are observed in recent literature reviews (BECKER, 2015; LARÉDO; KÖHLER; RAMMER, 2016). The extent of these positive input effects varies according to the country, estimation method and model specification. Table 1 lists some relevant studies consulted for this review.

TABLE 1
Overview of some key contributions on input additionality of R&D tax incentives

This evidence is the main source of information that supports a framework for design and implement policies to knowledge generation in latecomers, even if the obstacles in the transposition of these instruments are recognized. This overview reveals some caveats, when moving from these results to policy. First, these incentives are usually implemented at the national level in the context of national taxation laws, which restricts the evaluation of its effect under the specific legislative situation in that country. Second, the tax incentive is not a guarantee of effects on innovation as firms can re-label existing activities as R&D to take advantage of tax credits or just expand very low-quality R&D projects (DECHEZLEPRÊTRE et al., 2016DECHEZLEPRÊTRE, A. et al. Do tax incentives for research increase firm innovation? An RD design for R&D, 2016. (NBER Working Paper Series, n. 22405).). Regarding the latter limitation, evaluations of input additionality - the effect of the tax incentive on increasing private R&D expenditure - must be supplemented with output additionality - the contribution to tax incentives on innovation and economic impact.6 6 Despite the scant evaluation of output additionality, the topic has received increasing attention from the literature in the recent period (LARÉDO; KÖHLER; RAMMER, 2016).

Evaluations of the effects of R&D tax incentives on output additionality show that these instruments increase the probability of introducing new-to-the-firm products and new processes (CAPPELEN; RAKNERUD; RYBALKA, 2012CAPPELEN, Å.; RAKNERUD, A.; RYBALKA, M. The effects of R&D tax credits on patenting and innovations. Research Policy, v. 41, n. 2, p. 334-345, 2012.); new-to-the-market products (FALK, 2009FALK, R. The Coherence of the Instrument Mix. Report: Evaluation of Government Funding in RTDI from a Systems Perspective in Austria. Vienna: Austrian Institute of Economic Research, 2009.); and new-to-the-world and a new-to-the-market product (CZARNITZKI; HANEL; ROSA, 2011CZARNITZKI, D.; HANEL, P.; ROSA, J.M. Evaluating the impact of R&D tax credits on innovation: A microeconometric study on Canadian firms. Research Policy, v. 40, n. 2, p. 217-229, 2011.). Positive effects of tax incentives on output additionality are also found when measured by the turnover from new products (FREITAS et al., 2017FREITAS, I.B. et al. Sectors and the additionality effects of R&D tax credits: A cross-country microeconometric analysis. Research Policy, v. 46, n. 1, p. 57-72, 2017.); patenting value (DECHEZLEPRÊTRE et al., 2016DECHEZLEPRÊTRE, A. et al. Do tax incentives for research increase firm innovation? An RD design for R&D, 2016. (NBER Working Paper Series, n. 22405).); and flow of patents in the short and the long run (WESTMORE, 2013WESTMORE, B. R&D, Patenting and Growth: The Role of Public Policy. OECD, 2013. (OECD Economics Department Working Papers, n. 1047).).

In addition to the indirect incentives, many direct instruments for promoting private R&D fill the toolkit of policymakers and are related to a long tradition in public policies. The rationale for direct subsidies to private R&D is to stimulate innovation that leads to the production of new marketable products, processes, or services. The literature on the effectiveness of direct R&D support is extensive and covers a wide range of topics. One of the main surveys of the body of available studies accumulated over 35 years states that econometric evidence is ambivalent and there are crowding-out effects (DAVID; HALL; TOOLE, 2000DAVID, P.A.; HALL, B.H.; TOOLE, A.A. Is public R&D a complement or substitute for private R&D? a review of the econometric evidence. Research Policy, v. 29, n. 4-5, p. 497-529, 2000.). García-Quevedo (2004) meta-analysis finds similar results and argues that the relationship between R&D public funding and private R&D expenditure are difficult to reconcile due to its fundamentally empirical and descriptive nature.

However, the post-2000 empirical literature generally finds positive effects of public R&D subsidies on private R&D investment. One explanation for this revision is the effort to overcome the problems of sample selection bias using new econometric techniques for this purpose (BECKER, 2015BECKER, B. Public R&D Policies and Private R&D Investment: A Survey of the Empirical Evidence. Journal of Economic Surveys, v. 29, n. 5, p. 917-942, 2015.). Another possible explanation is the improvement in the effectiveness of this policy tool over time (KLETTE; MØEN, 2012KLETTE, T. J.; MØEN, J. R & D investment responses to R & D subsidies: A theoretical analysis and a microeconometric study. World Review of Science, Technology and Sustainable Development, v. 9, n. 2-4, p. 169-203, 2012.).

Accordingly, this growing literature finds additionality effects and rejects the hypothesis of crowding-out effects (BRONZINI; PISELLI, 2016BRONZINI, R.; PISELLI, P. The impact of R&D subsidies on firm innovation. Research Policy, v. 45, n. 2, p. 442-457, 2016.; CARBONI, 2017CARBONI, O.A. The effect of public support on investment and R&D: An empirical evaluation on European manufacturing firms. Technological Forecasting and Social Change, v. 117, n. C, p. 282-295, 2017.; CERULLI; POTÌ, 2012CERULLI, G.; POTÌ, B. The differential impact of privately and publicly funded R&D on R&D investment and innovation: The Italian case. Prometheus, v. 30, n. 1, p. 113-149, 2012.; CHOI; LEE, 2017LE, T.; JAFFE, A.B. The impact of R&D subsidy on innovation: evidence from New Zealand firms. Economics of Innovation and New Technology, v. 26, n. 5, p. 429-452, 2017.; HUERGO; TRENADO; UBIERNA, 2016HUERGO, E.; TRENADO, M.; UBIERNA, A. The impact of public support on firm propensity to engage in R&D: Spanish experience. Technological Forecasting and Social Change, v. 113, p. 206-219, 2016.; LE; JAFFE, 2017). A survey on the effect of public subsidies on firm R&D investment reveals limitation since the data most available come from studies performed in the short term, at the firm level and focused on the manufacturing sector (ZÚÑIGA-VICENTE et al., 2014ZÚÑIGA-VICENTE, J.Á. et al. Assessing the effect of public subsidies on firm R&D investment: A survey. Journal of Economic Surveys, v. 28, n. 1, p. 36-67, 2014.). A further limitation, and critical for the scope of this article, is that most of these studies are the result of experiments conducted in developed countries.7 7 For an expanded overview of this evidence, see Cunningham et al. (2016).

The scant evidence from developing countries shows that public support is an efficient tool to improve innovation performance of firms in Turkey and Poland (SZCZYGIELSKI et al., 2017SZCZYGIELSKI, K. et al. Does government support for private innovation matter? Firm-level evidence from two catching-up countries. Research Policy, v. 46, n. 1, p. 219-237, 2017.); generate significantly higher technological and commercialized innovation outputs in Small and Medium Enterprises (SMEs) in China (GUO; GUO; JIANG, 2016GUO, D.; GUO, Y.; JIANG, K. Government-subsidized R&D and firm innovation: Evidence from China. Research Policy, v. 45, n. 6, p. 1129-1144, 2016.); reduce the probability of abandoning an R&D project due to economic crisis of 2008 in Latin America (PAUNOV, 2012PAUNOV, C. The global crisis and firms’ investments in innovation. Research Policy, v. 41, n. 1, p. 24-35, 2012.). However, evidence from Brazil reveals that the adoption of a very wide variety of instruments could compromise the effectiveness of government support and raises doubts about the complementary character of government resources to support R&D (ROCHA, 2015ROCHA, F. Does governmental support to innovation have positive effect on R&D investments? Evidence from Brazil. Revista Brasileira de Inovação, v. 14, p. 37-60, 2015.).

The overview of evidence regarding fiscal incentives and direct support reveals that the effectiveness of these instruments can be enhanced when they are implemented in a coordinated manner since the former is more effective as a short-run intervention and the latter is more effective in medium to long-run strategies (BECKER, 2015BECKER, B. Public R&D Policies and Private R&D Investment: A Survey of the Empirical Evidence. Journal of Economic Surveys, v. 29, n. 5, p. 917-942, 2015.). Furthermore, despite the emphasis on the supply side of these instruments, it is necessary to recognize the complementarity exercised by policies on the demand side in the success of these interventions.

2.2 Policies for system failures

System failures occur when missing or weak connections (and synergies) among actors produce poor performance of an entire National Innovation System (NIS) (LEE, 2019LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.). According to the Schumpeterian tradition, this system is defined by firms, universities, public research laboratories, government agencies and financial institutions that interact in the generation, diffusion, and use of new and economically useful knowledge.

These system interactions have been extensively investigated in advanced countries but, still need to be fully understood in emerging and less developed countries, given the specifics of their firms and universities (ALBUQUERQUE et al., 2015ALBUQUERQUE, E. et al. Developing national systems of innovation: University-industry interactions in the global south. [s.l.] Edward Elgar Publishing Ltd., 2015.). In the initial stages of development, there is a lack of interaction between scientific and technological knowledge and institutions that catalyse this interaction are required to overcome specific thresholds (BERNARDES; ALBUQUERQUE, 2003BERNARDES, A.T.; ALBUQUERQUE, E.D.M.E. Cross-over, thresholds, and interactions between science and technology: Lessons for less-developed countries. Research Policy, v. 32, n. 5, p. 865-885, 2003.). Accordingly, given the immaturity of the NIS for the mutual transformation of scientific and technological knowledge, evidence shows that the latter - rather than the former - matters directly for economic growth in latecomers (KIM; LEE, 2015KIM, Y.K.; LEE, K. Different impacts of scientific and technological knowledge on economic growth: Contrasting science and technology policy in East Asia and Latin America. Asian Economic Policy Review, v. 10, n. 1, p. 43-66, 2015.).

The system failure is also described as the situation in which nations build up a certain level of capacity and the virtuous circle related to the functioning of dynamic complementarities does not work.8 8 In the same way that “institutional failures” can precede or reinforce the existence of “system failures”, which would require policies to correct this shortcoming. This arises from the mismatches or misalignments in the accumulation of tacit knowledge among the NIS agents, which result in an increase in their cognitive distance and a vicious circle of low interaction and learning (LEE, 2019LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.). To tackle this failure, policymakers suggest instruments both within an established system that is in the process of transformation and for an entirely new system. These instruments are designed to improve systemic capabilities and complementarities between the NIS components and aim to change their behaviour, encouraging more university-industry interactions, or between large and small firms (EDLER et al., 2016bEDLER, J. et al. Introduction: Making sense of innovation. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016b. p. 1-17.).

A systematic review of studies investigating public interventions aiming to enhance collaborative activities reveals a positive relationship between government subsidies and collaborative R&D between firms (CUNNINGHAM; GÖK, 2016CUNNINGHAM, P. et al. The innovation policy mix. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 505-542.). These studies are heterogeneous in terms of scope and results, but a careful analysis reveals three main groups. First, we find the studies that identify the effects of government subsidies on the design of collaborations and encourage firms to undertake domestic upstream and downstream partners (KANG; PARK, 2012KANG, K.N.; PARK, H. Influence of government R&D support and inter-firm collaborations on innovation in Korean biotechnology SMEs. Technovation, v. 32, n. 1, p. 68-78, 2012.); expand their external collaboration breadth (CHAPMAN; LUCENA; AFCHA, 2018CHAPMAN, G.; LUCENA, A.; AFCHA, S. R&D subsidies & external collaborative breadth: Differential gains and the role of collaboration experience. Research Policy, v. 47, n. 3, p. 623-636, 2018.); and work with a heterogeneous range of partners (AHN; LEE; MORTARA, 2020AHN, J. M.; LEE, W.; MORTARA, L. Do government R&D subsidies stimulate collaboration initiatives in private firms? Technological Forecasting & Social Change, v. 151, 2020.).

The second group of studies deals with the mismatches among the NIS agents and its results reveal that the supporting collaboration is associated with academic researchers’ industrial activity (BOZEMAN; GAUGHAN, 2007BOZEMAN, B.; GAUGHAN, M. Impacts of grants and contracts on academic researchers’ interactions with industry. Research Policy, v. 36, n. 5, p. 694-707, 2007.); the higher probability of firm cooperation with a public research organisation (BUSOM; FERNÁNDEZ-RIBAS, 2008BUSOM, I.; FERNÁNDEZ-RIBAS, A. The impact of firm participation in R&D programmes on R&D partnerships. Research Policy, v. 37, n. 2, p. 240-257, 2008.); and increase the rate of agreements between manufacturing firms and universities or technological centres (AFCHA CHÁVEZ, 2011AFCHA CHÁVEZ, S.M. Behavioural additionality in the context of regional innovation policy in Spain. Innovation: Management, Policy and Practice, v. 13, n. 1, p. 95-110, 2011.).

The third group of studies output effects and report the collaborative R&D on firm increasing R&D spending (ARRANZ; ARROYABE, 2008ARRANZ, N.; ARROYABE, J.C.F. de. The choice of partners in R&D cooperation: An empirical analysis of Spanish firms. Technovation, v. 28, n. 1-2, p. 88-100, 2008.); increasing the probability of patenting (CZARNITZKI; EBERSBERGER; FIER, 2007CZARNITZKI, D.; EBERSBERGER, B.; FIER, A. The relationship between R&D collaboration, subsidies and R&D performance: Empirical evidence from Finland and Germany. Journal of Applied Econometrics, University of Oxford. v. 22, p. 1347-1366, 2007.); and greater propensity to establish R&D agreements (SEGARRA-BLASCO; ARAUZO-CAROD, 2008SEGARRA-BLASCO, A.; ARAUZO-CAROD, J. M. Sources of innovation and industry-university interaction: Evidence from Spanish firms. Research Policy, v. 37, n. 8, p. 1283-1295, 2008.). The increase in spending also generates spillovers effects, given the increase in R&D spending in other firms (WATANABE; KISHIOKA; NAGAMATSU, 2004WATANABE, C.; KISHIOKA, M.; NAGAMATSU, A. Effect and limit of the government role in spurring technology spillover - A case of R&D consortia by the Japanese government. Technovation, v. 24, n. 5, p. 403-420, 2004.). According to Hottenrott and Lopes-Bento (2014HOTTENROTT, H.; LOPES-BENTO, C. (International) R&D collaboration and SMEs: The effectiveness of targeted public R&D support schemes. Research Policy, v. 43, n. 6, p. 1055-1066, 2014.), public subsidies are even more effective in generating R&D spending in cases of international collaboration. Caloffi et al (2018CALOFFI, A. et al. A comparative evaluation of regional subsidies for collaborative and individual R&D in small and medium-sized enterprises. Research Policy, v. 47, n. 8, p. 1437-1447, 2018.) show that the expected results of public subsidies over time depend on the characteristics of the beneficiaries. SMEs with some prior R&D experience increase the amount of spontaneous R&D investment and firms with modest R&D experience produce an increasing number of R&D-performing SMEs.

2.3 Policies for size failure

The lack of world-class businesses in developing countries is conceptually defined as “size failure”. Within these contexts, the market space that would be occupied by large companies (as in developed countries) is filled by small and medium-sized firms, which are perceived as an insufficient organizational form in leading a transition from middle- to high-income status. The Schumpeterian legacy states that large and quasi-monopolistic firms are the businesses model capable to exploit economies of scale and scope. Thus, the need for large companies in this transition is due to their ability to achieve high levels of knowledge production and capacity to undertake R&D and marketing activities with higher value-added (LEE, 2019LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.).

Despite this rationale, the benefits of big business are a controversial topic and its results in terms of social welfare are far from a consensus among scholars. For instance, Fogel, Uyarra and Larangja (2008FOGEL, K.; MORCK, R.; YEUNG, B. Big business stability and economic growth: Is what’s good for General Motors good for America? Journal of Financial Economics, v. 89, n. 1, p. 83-108, 2008.) find faster economic growth in countries where big business is less stable over time. Moreover, the big business stability occurs where governments are larger, civil codes hold sway, red tape is denser, banks are more dominant, and the economy is less open to global.

A critical issue in this debate is the lack of a well-established toolkit for this policy goal, which results in a relative scarcity of evaluations of its effectiveness. Generally, the instruments to encourage the growth and consolidation of large companies are designed and implemented within the scope of trade, competition and regulations policies and their public intentionality is not observed. For this article, this section reviews the effectiveness of policies designed to building capacity, especially in promoting R&D through fiscal incentives and direct support.

Guceri (2013GUCERI, I. Tax incentives and R&D: An evaluation of the 2002 UK reform using micro data. 2013. (University of Oxford Working paper).) study on R&D tax incentive scheme in the UK shows that large firms increased their R&D spending by more than an additional 18% in comparison to the SME control group. In a similar study, Bond and Guceri (2012BOND, S.; GUCERI, I. Trends in UK BERD after the Introduction of R&D Tax Credits. Oxford University Center for Business Taxation, 2012. (Working Paper, n. 12/01).) found effects of the UK R&D Tax Relief for large firms in increasing the intensity of R&D, albeit almost wholly concentrated in high tech sub-sectors of manufacturing.

The effectiveness of R&D tax incentive schemes requires special attention when the time dimension is included in the analysis since the effects and outcomes of these activities are long-run. Lokshin and Mohnen (2012LOKSHIN, B.; MOHNEN, P. How effective are level-based R&D tax credits? Evidence from the Netherlands. Applied Economics, v. 44, n. 12, p. 1527-1538, 2012.) examine the effects of the Dutch scheme and find that input additionality diminishes over time, hence the crowding out of private R&D can only be averted for small firms, while the scheme becomes ineffective in large firms.

As in the case of fiscal incentives, the evidence of direct support is mixed and not clear-cut. According to some authors, the additionality effect is observed differently depending on the size of the company. Lach (2002LACH, S. Do R&D subsidies stimulate or displace private R&D? Evidence from Israel. Journal of Industrial Economics, v. 50, n. 4, p. 369-390, 2002.) - using data from the Israeli manufacturing sector - shows increased R&D spending for small firms and decreases for large firms. Lenihan and Hart (2006LENIHAN, H.; HART, M. Evaluating the additionality of public sector assistance to irish firms: A question of ownership? Policy Studies, v. 27, n. 2, p. 115-133, 2006.) developed an approach to estimate the net additionality and found a higher deadweight for larger domestic firms, that is, the investments on R&D would have occurred regardless of subsidies.

Another stream of research reports positive effects of direct support for large firms, although the effect on R&D intensity is more significant for small firms (PAUNOV, 2012PAUNOV, C. The global crisis and firms’ investments in innovation. Research Policy, v. 41, n. 1, p. 24-35, 2012.). Cerulli and Poti (2012CERULLI, G.; POTÌ, B. The differential impact of privately and publicly funded R&D on R&D investment and innovation: The Italian case. Prometheus, v. 30, n. 1, p. 113-149, 2012.) in a firm-level study suggests that the Italian R&D policy instrument has been successful in promoting both input and output additionality and firms with higher performances are generally larger.

In cases where the policy outcomes are measured by new products and services, Hujer and Radic´ (2005HUJER, R.; RADIĆ, D. Evaluating the impacts of subsidies on innovation activities in Germany. Scottish Journal of Political Economy, v. 52, n. 4, p. 565-586, 2005.) study of the German Federal Employment Office shows a significant effect for large firms. Herrera and Bravo Ibarra (2010HERRERA, L.; BRAVO IBARRA, E. R. Distribution and effect of R&D subsidies: A comparative analysis according to firm size. Intangible Capital, v. 6, n. 2, p. 272-299, 2010.), when comparing R&D subsidies according to firm size, conclude that large firms only show a positive and significant effect on in-house technology generation and R&D subsidies have a positive, significant effect only on the tendency to patenting of large firms.

3. Summary and discussion

Arguments in defence of evidence-based policies have proliferated in the past decades and have had a deep impact on government bureaucracies, academic institutions, and the media. According to this movement, evidence matters for public policymaking and perform a critical role in demonstrating “what works”. However, science-based policies are threatened by the “politicisation of science”, where political interests appear to drive the corrupt to shape or cherry-picking the evidence (PARKHURST, 2017PARKHURST, J.O. The politics of evidence: from evidence-based policy to the good governance of evidence. Abingdon; New York: Routledge, 2017.). Hence, scholars have proposed a toolkit that aims to guide policymakers in various aspects of interventions, based on existing evidence (BLOOM; VAN REENEN; WILLIAMS, 2019BLOOM, N.; VAN REENEN, J.; WILLIAMS, H. A toolkit of policies to promote innovation. Journal of Economic Perspectives, v. 33, n. 3, p. 163-184, 2019.).

Despite this apparent consensus, another strand of academics is critical of the notion that public policymaking can be based on evidence alone. These critical voices argue that social policies involve trade-offs between several competing social values and only a minor group of political decisions can be limited to decisions based exclusively on technical evidence of the effects of interventions. In contrast, these scholars are worried about the “depoliticisation of politics”, given that the emphasis on evidence-based can obscure or marginalize social values (PARKHURST, 2017PARKHURST, J.O. The politics of evidence: from evidence-based policy to the good governance of evidence. Abingdon; New York: Routledge, 2017.).

In terms of policies for the catch-up, although practices in forerunners usually provide a model, the needs of latecomers countries inevitably differ in various and important ways from the existing templates (MAZZOLENI; NELSON, 2007MAZZOLENI, R.; NELSON, R.R. Public research institutions and economic catch-up. Research Policy, v. 36, n. 10, p. 1512-1528, 2007.). This can be illustrated by the paradox that states “to be similar, you’ve got to be different” (LEE, 2019LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.) which means that long-term success requires a different path from that adopted by developed countries.

According to Parkhurst (2017PARKHURST, J.O. The politics of evidence: from evidence-based policy to the good governance of evidence. Abingdon; New York: Routledge, 2017.), these views are not seen as mutually exclusive since they are based on normative rather than on epistemological differences. Thus, a pragmatic view tends to recognize that both sets of values are important goals to deal with initiatives to improve the use of evidence within the catch-up process.

Despite the deeper implications of both sets of values, a first conclusion from the extant literature is that much of the evidence is invariably context-dependent, and lessons for latecomers contexts need to be in perspective. However, this fact does not prevent this evidence from being a basis for guiding the efforts for catch-up policy efforts in latecomers. Table 2 synthesizes these judgements according to the failure and reveals the identified variations in terms of quality and conclusiveness.

TABLE 2
Summary of instruments and evidence judgements

Policies to deal with capability failure are the interventions most supported by evidence from policy instruments. Evaluations show that R&D tax incentives and direct funding increased business R&D expenditure over time (input additionality). There are positive effects of R&D tax incentive on output additionality, especially for products and processes, but the evidence for direct funding is limited and contradictory. In both cases, these effects vary greatly depending on the country, sectors, period and method used. Therefore, this context-dependent characteristic of the instruments makes careful examination necessary before their implementation in latecomers since most of the evidence comes from developed countries.

System failures are overcome by instruments such as subsidies to collaborative R&D and reviewed studies present significant evidence of their effectiveness. In terms of input additionality, government subsidies have strong effects on collaborative R&D between firms. The evaluations also report a wide range of output additionality from these subsidies, which are broadly defined and measured. Policymaking based on this evidence requires caution since the evidence comes from developed innovation systems and the effects are limited to the target group, not to broader populations of firms.

The instruments to tackle the size failure are the most contradictory among the groups investigated. Despite some studies finding positive input additionality, most reveal that fiscal incentives and direct support to large firms become ineffective when the time dimension is included. These results impose a dilemma for policymakers since small firms seem to respond more positively to government support than large companies (CRISCUOLO et al., 2019CRISCUOLO, C. et al. Some causal effects of an industrial policy. American Economic Review, v. 109, n. 1, p. 48-85, 2019.). However, the focus on small firms can limit the emergence of large companies since entrepreneurs are discouraged from expanding beyond thresholds that disqualify them from these subsidies (BLOOM; VAN REENEN; WILLIAMS, 2019BLOOM, N.; VAN REENEN, J.; WILLIAMS, H. A toolkit of policies to promote innovation. Journal of Economic Perspectives, v. 33, n. 3, p. 163-184, 2019.).

The R&D activities undertaken by large firms when compared to SME have characteristics that change their performance in response to government support. First, since R&D is characterized by indivisibilities and minimum size, projects of all types and complexities are more likely to become feasible in the context of large firms. Second, R&D activities also involve high fixed costs and large firms are better able to absorb the spreading effects of fixed costs due to their larger scale of production. Third, large firms are more capable to capture the spillovers from R&D activities given the effectiveness of protecting their intellectual property. Ultimately, large firms are endowed with a structure that enables them to secure funding for risky projects given capital market imperfections.

3.1 Interaction between instruments

This summary leads us to question the consequences of the interplay between instruments and their effects on the outcomes of the policy intervention. The question that arises is whether there are effects from the combination of policy instruments, since interactions can turn into “complementarities”, when the presence of one instrument increases the effectiveness of another, or “trade-offs”, where one instrument attenuates the influence of another (EDLER et al., 2016cEDLER, J. et al. Conclusions: Evidence on the effectiveness of innovation policy intervention. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016c. p. 543-564.).

During policy enforcement, it is evident to assume that both possibilities for interaction often occur. Meanwhile, the evaluations of policy instruments are largely done in isolation and extant evidence on the interplay of policy instruments is extremely rare. Accordingly, little efforts have been devoted to producing meta-evaluation of these country evaluations. A comparative study commissioned by the European Commission’s Research Directorate-General found no analysis concerning the overall effects of policy mixes and interplay of instruments (CUNNINGHAM et al., 2016CUNNINGHAM, P. et al. The innovation policy mix. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 505-542.).

The extent to which these instruments can achieve synergies and positive complementarities and the mechanisms to minimise negative interactions between instruments are a topic with increasing attention among policymakers. The discussion on this phenomenon has been organized around the so-called “policy mix” and refers to the combination of policy instruments which interact to influence the quantity and quality of interventions, both in public and private sectors.

The main idea is that a policy mix is more than simply a portfolio of instruments and the appropriate understanding of the evolution of policy mixes is a precondition to any evaluation. The conceptual mechanism proposed by Flanagan, Uyarra and Larangja (2010FLANAGAN, K.; UYARRA, E.; LARANGJA, M. The “policy mix” for innovation: Rethinking innovation policy in a multi-level, multi-actor context. Manchester Institute of Innovation, 2010. (Research Working Paper Series, n. 599).) argues that policy mix interactions can occur across the policy environment, which is embodied by different policy sub-systems, between different levels of governance, across geographical space and over time. From this framework, the policymakers take advantage of potential complementarities or address potential sources of tension by efforts at improved coordination or design a coherent policy mix.

4. Challenges and lessons for evaluations

A usual conclusion in the reviewed studies on the effectiveness of instruments is that “the evidence is mixed”. Some argue that when we recognize the complex and multidimensional nature of the catch-up process, it should not be surprising that quantitative evaluations achieve this type of results. However, a critical reflection on the limits and possibilities of contemporary methods that are emerging could guide the future directions of evaluations.

A challenge of evaluations is the adequate treatment of causal interpretations. Usually, the policies for catch-up are based on a relevant causal argument that there is a better outcome that would not have occurred in the absence of the intervention. Although it appears this would be fertile ground for causal inference, the review shows that this has not been the case until recently. Nevertheless, causal inference has been at the centre of the concerns of quantitative policy assessment in several areas. The counterfactual evaluation aims to estimate the magnitude of this specified causal effect, using appropriate econometric techniques.

The use of counterfactual in policies has been the subject of disagreement among scholars. Some believe that steps to build this counterfactual are excessively arbitrary and suggest that analysis is impossible. According to others, using contemporary econometric tools, questions and research designs, it is possible to conduct useful counterfactuals comparisons for policies (LANE, 2020LANE, N. The New Empirics of Industrial Policy. Journal of Industry, Competition and Trade, n. 2010, p. 4041-4042, 2020.). In other words, the counterfactual evaluation methods cannot answer many relevant questions since that important effects cannot be quantified, however, still many issues for which causal inference would be feasible and useful (BRAVO-BIOSCA, 2019BRAVO-BIOSCA, A. Experimental Innovation Policy, 2019. (NBER Working Paper, n. 26273).).

In the case of policies to knowledge generation, a relevant question in the counterfactual language is: “Would firms underinvest in knowledge production, or be less innovative, if they had not received a public incentive?” Since most of these incentives are related to R&D promotion, one of the challenges to be overcome in the evaluations is lacking experimental evidence in this area.

This experiment-based evaluation format is contextualized in a research environment increasingly dominated by randomized policy evaluations (DUFLO; GLENNERSTER; KREMER, 2007DUFLO, E.; GLENNERSTER, R.; KREMER, M. Using Randomization in Development Economics Research: A Toolkit. In: SCHULTZ, T.; STRAUSS, J. Handbook of Development Economics. [s.l.] Elsevier, 2007. v. 4. p. 3895-3962.). To exemplify this method, we can imagine randomly partitioning a population of firms into a treatment and a control group. Whereas the former receives financial support and to latter funding is denied, the differences in performance between the two groups could be directly attributable to the grant. The main obstacle to this experiment is that policymakers and taxpayers are unwilling, without extensive due diligence, to randomly handing amounts of financial incentives among firms (HÜNERMUND; CZARNITZKI, 2019aHÜNERMUND, P.; CZARNITZKI, D. Innovation Policy. ifo DICE Repor, v. 17, p. 37-54, 2019a.). Furthermore, the application of this method is even more complicated in developing countries, given that the alleged “randomness” in the selection of beneficiaries tends to be difficult or unviable, considering the lack of a sufficient number of firms of different industries and sizes. Alternatively, the evaluations are conducted mostly based on data observed ex-post, collected from subsidy schemes where firms have been hand-selected from a list of applicants. This method renders the econometric policy evaluation task even more complex.

Attempts to overcome these obstacles arise when it is recognized that these policies can be employed more effectively, when it is known whether the combined net effects of treatment exceed those of the combined costs of the treatment. Although the quantitative analysis of net effects of interventions can be particularly useful for policymakers, studies demonstrating these effects are extremely rare.

Ex-post policy evaluation also needs to handle technical difficulties such as the so-called confounding problem (BAREINBOIM; PEARL, 2016BAREINBOIM, E.; PEARL, J. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences of the United States of America, v. 113, n. 27, p. 7345-7352, 2016.). Following the reasoning of the previous example, differences between funded and non-funded firms resulting from the selection process, if not carefully considered, can significantly affect the outcomes of the evaluation. However, a successful assessment requires that all confounders be observed, and as can be imagined, this rarely occurs. For instance, the survey on the effectiveness of public subsidies of private R&D spending shows that very few studies adequately take the confounding problem into account (ZÚÑIGA-VICENTE et al., 2014ZÚÑIGA-VICENTE, J.Á. et al. Assessing the effect of public subsidies on firm R&D investment: A survey. Journal of Economic Surveys, v. 28, n. 1, p. 36-67, 2014.).

The evaluation of a policy tool without controlling for simultaneous public programmes aiming at the same objective can also result in procedural confounding due to hidden treatments (GUERZONI; RAITERI, 2012GUERZONI, M.; RAITERI, E. Innovative Procurement and R&D Subsidies: hidden treatment and new empirical evidence on the technology policy mix in a quasi-experimental setting. Bureau of Research in Innovation, Complexity and Knowledge, n. 18, p. 1-32, 2012.). In other words, the results of the target treatment are distorted when only one particular treatment is observed without controlling for other treatments. To handle these problems, emerging studies have proposed the use of rankings, thresholds, and Regression Discontinuity Design (RDD) to provide more convincing causal evidence (BRONZINI; PISELLI, 2016BRONZINI, R.; PISELLI, P. The impact of R&D subsidies on firm innovation. Research Policy, v. 45, n. 2, p. 442-457, 2016.; DECHEZLEPRÊTRE et al., 2016DECHEZLEPRÊTRE, A. et al. Do tax incentives for research increase firm innovation? An RD design for R&D, 2016. (NBER Working Paper Series, n. 22405).; HOWELL, 2017HOWELL, S. T. Financing innovation: Evidence from R&D grants. American Economic Review, v. 107, n. 4, p. 1136-1164, 2017.; HÜNERMUND; CZARNITZKI, 2019bHÜNERMUND, P.; CZARNITZKI, D. Estimating the causal effect of R&D subsidies in a pan-European program. Research Policy, v. 48, n. 1, p. 115-124, 2019b.). Therefore, research designs are planned in detail to confront endemic endogeneity and tackle the confounding problem.

Since these frontiers of evaluation tend to be undertaken at the level of specific policy programs and schemes, challenges for their use during the catch-up process become evident. A first challenge is imposed by the system complexity, their multiple interactions over time and, essentially, the needed data to support assessments.9 9 An example is the methods to measure and compare costs and benefits over time. The current toolkit makes it difficult to capture both sides of the cost-benefit ledger since many of the benefits are based on future hopes for benefits and are fully captured only in the long run. The complexity is given by the fact that these systems combine heterogeneous institutions such as universities, technology parks, clusters, business incubators, startups and others. To overcome these obstacles, governments should become more open to the use of experiments in policy evaluations and there is a need to increase cooperation with researchers.

4.1 Sequence and context of policies

This article advocates that evaluation of policies for knowledge generation in latecomers could improve interventions to deal with failures during the catch-up process. Therefore, it needs to consider the effectiveness of a range of instruments to achieve an increasing improvement in the design of the intervention. This presupposes learning about these interventions and the capacity to capture and interpret their effects. Yet, the efficiency of these instruments is a necessary but not sufficient condition to guarantee the effectiveness of policies and since the evidence is invariably context-dependent, caution is needed when drawing lessons for latecomers’ contexts.

According to the current literature, evaluations have led to a perspective that isolates individual instruments from their overall policy context. Since the instruments do not emerge in a vacuum, studies increasingly need to demonstrate how contexts can affect the overall results. The interventions need to put into perspective the existence of unobservable forces such as government capacity, market imperfections, and the targets of policymakers.10 10 This aspect is notably relevant for most developing countries that tend to suffer more severely from economic downturns driven by local and international issues and obstacles to building coalitions around common goals given the uneven social structure. These forces can confound the relationship between instrument performance and policy interventions. For instance, the empirical relationship between private R&D spending and endogenous politics cannot distinguish between the impact of the intervention and unobserved political forces behind the policy. In the absence of context analysis, it is difficult to determine whether the evaluation is testing the practical aspects of the policies per se or the interaction of these policies with broader political forces.11 11 See Rodrik (2012) for a detail discussion of these evaluation failures in the industrial policies context.

The role of the political context requires attention since technocratic policies may not be as salient in the time frame of evaluation. While instruments such as tax incentives are well established in terms of evidence, this is not the case for other incentive, such as incentives for building capabilities which, given their nature, their effects may not be properly captured by evaluations. In some cases, the measure of the impact requires more granular evidence and richer institutional context than the present generation of empirical studies.

The political context is also crucial since the instruments are embedded in institutions that shape their performance over time. Suzigan, Garcia and Feitosa (2020SUZIGAN, W.; GARCIA, R.; FEITOSA, P.H. A. Institutions and industrial policy in Brazil after two decades: have we built the needed institutions? Economics of Innovation and New Technology, v. 29, n. 7, p. 799-813, 2020.) demonstrate how the general political context was decisive in determining the results of policies to encourage the generation of knowledge in Brazil (based on R&D indicators). Thus, evaluations of instruments need to consider the intention of policy and recognize the existence of unobserved forces that shape the scope of the policy.

A further challenge arises when the evidence is properly captured only as a sequence of events is observed. The incentives for knowledge generation are highly dependent on the time dimension and the verification of many outcomes depends on the succession of different instruments over a significant period. A study on the Danish and Norwegian wind industry reveals a cumulative impact on permanent technical change, when different policy instruments are coordinated over time (BUEN, 2006BUEN, J. Danish and Norwegian wind industry: The relationship between policy instruments, innovation and diffusion. Energy Policy, v. 34, n. 18, p. 3887-3897, 2006.).12 12 For discussion of sequencing policy interventions in the case of energy agencies, see Cunningham et al. (2016). This study shows that the cumulative impact of the interventions is influenced by the succession of different instruments designed for specific needs and it is captured by the combination of supply and demand measures. In other words, the evaluation of incentives to knowledge generation must be followed up by further tailored supply or demand measures.

When considering the temporal dimension, a further difficulty stems from the intrinsically “dynamic” character of technological trajectories that require constant reorientation of these interventions. When it is recognized that the evidence on these interventions tend to be nuanced and pose challenges for emerging evaluation methods, a possible consequence is that methodological advances are to render a large portion of current methods impotent. These limits and shortcomings have led scholars to reinforce the importance of qualitative reviews, among conventional quantitative analyses when seeking to capture the effectiveness of instruments in policies such as catch-up.13 13 In addition, while the analysis of the results is sufficient to assess the “efficiency” of these policies, the “effectiveness” of the catching-up process involves some structural changes that may require even more complex framework and instruments.

Considering the proliferation of new studies and methods, the combination of approaches can prove especially useful to deal with the challenges discussed. In particular, the studies focus on specific case studies and natural experiences to estimate the impact of catch-up policies. Although there are limitations to this approach, it can still provide valuable information about these policies. An example is the return of studies that adopt traditional historical investigations that rigorously detail specific political cases, institutional schemes, and specific projects (LANE, 2020LANE, N. The New Empirics of Industrial Policy. Journal of Industry, Competition and Trade, n. 2010, p. 4041-4042, 2020.). Since the successful upward transition rarely occurs, this approach allows the extraction of empirical lessons from specific episodes and is crucial for shaping the way we understand the complex relationships between government action and its broader effects.

5. Concluding remarks

This study proposed a review of studies about the effectiveness of policies for knowledge generation and discusses the potential of evaluating these instruments to address failures in the catch-up process. The results show that instruments for tackle failures are heterogeneous and the availability and accuracy of evidence vary significantly. The policies for capability failure and system failure are interventions that are more evidence-based and less controversial than policies for size failure. Moreover, much of the evidence report conflicting results and can lead to misunderstandings about the potential of these interventions. The lessons for latecomers are that evaluations tend to isolate individual instruments from their overall policy context and given its characteristic of strong dependence on the context, caution is needed when drawing lessons for policies.

Emerging studies and their attempts to adequately address the problems of causal evidence, endogeneity and confounding problem are relevant initiatives for the improvement of evaluation of catch-up policies. However, these evaluations require more granular evidence which poses challenges for methods of analysis. This article argues that an adequate understanding of catch-up policies also requires detailed particular political cases, institutional schemes, and specific projects. Furthermore, this understanding requires the recognition of the essentially complex nature of catching-up processes in developing countries. In general, our understanding of these interventions is enhanced if emerging approaches have been enriched by specific policy case studies, institutional details, and policy contexts. Put differently, the combination of approaches, with appropriate caveats, represent a significant potential to improve the future production of evidence of policies for the catch-up and learning about policy design.

References

  • ABRAMOVITZ, M. Catching Up, Forging Ahead, and Falling Behind. The Journal of Economic History, v. 46, n. 2, p. 385-406, 1986.
  • AFCHA CHÁVEZ, S.M. Behavioural additionality in the context of regional innovation policy in Spain. Innovation: Management, Policy and Practice, v. 13, n. 1, p. 95-110, 2011.
  • AHN, J. M.; LEE, W.; MORTARA, L. Do government R&D subsidies stimulate collaboration initiatives in private firms? Technological Forecasting & Social Change, v. 151, 2020.
  • ALBUQUERQUE, E. et al. Developing national systems of innovation: University-industry interactions in the global south. [s.l.] Edward Elgar Publishing Ltd., 2015.
  • ARRANZ, N.; ARROYABE, J.C.F. de. The choice of partners in R&D cooperation: An empirical analysis of Spanish firms. Technovation, v. 28, n. 1-2, p. 88-100, 2008.
  • BAREINBOIM, E.; PEARL, J. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences of the United States of America, v. 113, n. 27, p. 7345-7352, 2016.
  • BECKER, B. Public R&D Policies and Private R&D Investment: A Survey of the Empirical Evidence. Journal of Economic Surveys, v. 29, n. 5, p. 917-942, 2015.
  • BERNARDES, A.T.; ALBUQUERQUE, E.D.M.E. Cross-over, thresholds, and interactions between science and technology: Lessons for less-developed countries. Research Policy, v. 32, n. 5, p. 865-885, 2003.
  • BLOOM, N.; VAN REENEN, J.; WILLIAMS, H. A toolkit of policies to promote innovation. Journal of Economic Perspectives, v. 33, n. 3, p. 163-184, 2019.
  • BOND, S.; GUCERI, I. Trends in UK BERD after the Introduction of R&D Tax Credits. Oxford University Center for Business Taxation, 2012. (Working Paper, n. 12/01).
  • BOZEMAN, B.; GAUGHAN, M. Impacts of grants and contracts on academic researchers’ interactions with industry. Research Policy, v. 36, n. 5, p. 694-707, 2007.
  • BRAVO-BIOSCA, A. Experimental Innovation Policy, 2019. (NBER Working Paper, n. 26273).
  • BRONZINI, R.; PISELLI, P. The impact of R&D subsidies on firm innovation. Research Policy, v. 45, n. 2, p. 442-457, 2016.
  • BUEN, J. Danish and Norwegian wind industry: The relationship between policy instruments, innovation and diffusion. Energy Policy, v. 34, n. 18, p. 3887-3897, 2006.
  • BUSOM, I.; FERNÁNDEZ-RIBAS, A. The impact of firm participation in R&D programmes on R&D partnerships. Research Policy, v. 37, n. 2, p. 240-257, 2008.
  • CALOFFI, A. et al. A comparative evaluation of regional subsidies for collaborative and individual R&D in small and medium-sized enterprises. Research Policy, v. 47, n. 8, p. 1437-1447, 2018.
  • CAPPELEN, Å.; RAKNERUD, A.; RYBALKA, M. The effects of R&D tax credits on patenting and innovations. Research Policy, v. 41, n. 2, p. 334-345, 2012.
  • CARBONI, O.A. The effect of public support on investment and R&D: An empirical evaluation on European manufacturing firms. Technological Forecasting and Social Change, v. 117, n. C, p. 282-295, 2017.
  • CASTELLACCI, F. Closing the Technology Gap? Review of Development Economics, v. 15, n. 1, p. 180-197, 2011.
  • CERULLI, G.; POTÌ, B. The differential impact of privately and publicly funded R&D on R&D investment and innovation: The Italian case. Prometheus, v. 30, n. 1, p. 113-149, 2012.
  • CHAPMAN, G.; LUCENA, A.; AFCHA, S. R&D subsidies & external collaborative breadth: Differential gains and the role of collaboration experience. Research Policy, v. 47, n. 3, p. 623-636, 2018.
  • CHOI, J.; LEE, J. Repairing the R&D market failure: Public R&D subsidy and the composition of private R&D. Research Policy, v. 46, n. 8, p. 1465-1478, 2017.
  • CIMOLI, M.; PEREIMA, J.B.; PORCILE, G. A technology gap interpretation of growth paths in Asia and Latin America. Research Policy, v. 48, n. 1, p. 125-136, 2019.
  • COAD, A. Persistent heterogeneity of R&D intensities within sectors: Evidence and policy implications. Research Policy, v. 48, n. 1, p. 37-50, 2019.
  • CRISCUOLO, C. et al. Some causal effects of an industrial policy. American Economic Review, v. 109, n. 1, p. 48-85, 2019.
  • CUNNINGHAM, P.; GÖK, A. The impact of innovation policy schemes for collaboration. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 239-278.
  • CUNNINGHAM, P. et al. The innovation policy mix. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 505-542.
  • CZARNITZKI, D.; EBERSBERGER, B.; FIER, A. The relationship between R&D collaboration, subsidies and R&D performance: Empirical evidence from Finland and Germany. Journal of Applied Econometrics, University of Oxford. v. 22, p. 1347-1366, 2007.
  • CZARNITZKI, D.; HANEL, P.; ROSA, J.M. Evaluating the impact of R&D tax credits on innovation: A microeconometric study on Canadian firms. Research Policy, v. 40, n. 2, p. 217-229, 2011.
  • DAVID, P.A.; HALL, B.H.; TOOLE, A.A. Is public R&D a complement or substitute for private R&D? a review of the econometric evidence. Research Policy, v. 29, n. 4-5, p. 497-529, 2000.
  • DECHEZLEPRÊTRE, A. et al. Do tax incentives for research increase firm innovation? An RD design for R&D, 2016. (NBER Working Paper Series, n. 22405).
  • DUFLO, E.; GLENNERSTER, R.; KREMER, M. Using Randomization in Development Economics Research: A Toolkit. In: SCHULTZ, T.; STRAUSS, J. Handbook of Development Economics. [s.l.] Elsevier, 2007. v. 4. p. 3895-3962.
  • DUGUET, E. The effect of the incremental R&D tax credit on the private funding of R&D an econometric evaluation on french firm level data. Revue d’Economie Politique, v. 122, n. 3, p. 405-435, 2012.
  • EDLER, J. et al. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016a.
  • EDLER, J. et al. Introduction: Making sense of innovation. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016b. p. 1-17.
  • EDLER, J. et al. Conclusions: Evidence on the effectiveness of innovation policy intervention. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016c. p. 543-564.
  • FALK, R. The Coherence of the Instrument Mix. Report: Evaluation of Government Funding in RTDI from a Systems Perspective in Austria. Vienna: Austrian Institute of Economic Research, 2009.
  • FLANAGAN, K.; UYARRA, E.; LARANGJA, M. The “policy mix” for innovation: Rethinking innovation policy in a multi-level, multi-actor context. Manchester Institute of Innovation, 2010. (Research Working Paper Series, n. 599).
  • FOGEL, K.; MORCK, R.; YEUNG, B. Big business stability and economic growth: Is what’s good for General Motors good for America? Journal of Financial Economics, v. 89, n. 1, p. 83-108, 2008.
  • FREITAS, I.B. et al. Sectors and the additionality effects of R&D tax credits: A cross-country microeconometric analysis. Research Policy, v. 46, n. 1, p. 57-72, 2017.
  • GARCÍA-QUEVEDO, J. Do public subsidies complement business R&D? A meta-analysis of the econometric evidence. Kyklos, v. 57, n. 1, p. 87-102, 2004.
  • GUCERI, I. Tax incentives and R&D: An evaluation of the 2002 UK reform using micro data. 2013. (University of Oxford Working paper).
  • GUERZONI, M.; RAITERI, E. Innovative Procurement and R&D Subsidies: hidden treatment and new empirical evidence on the technology policy mix in a quasi-experimental setting. Bureau of Research in Innovation, Complexity and Knowledge, n. 18, p. 1-32, 2012.
  • GUO, D.; GUO, Y.; JIANG, K. Government-subsidized R&D and firm innovation: Evidence from China. Research Policy, v. 45, n. 6, p. 1129-1144, 2016.
  • HERRERA, L.; BRAVO IBARRA, E. R. Distribution and effect of R&D subsidies: A comparative analysis according to firm size. Intangible Capital, v. 6, n. 2, p. 272-299, 2010.
  • HOTTENROTT, H.; LOPES-BENTO, C. (International) R&D collaboration and SMEs: The effectiveness of targeted public R&D support schemes. Research Policy, v. 43, n. 6, p. 1055-1066, 2014.
  • HOWELL, S. T. Financing innovation: Evidence from R&D grants. American Economic Review, v. 107, n. 4, p. 1136-1164, 2017.
  • HUERGO, E.; TRENADO, M.; UBIERNA, A. The impact of public support on firm propensity to engage in R&D: Spanish experience. Technological Forecasting and Social Change, v. 113, p. 206-219, 2016.
  • HUJER, R.; RADIĆ, D. Evaluating the impacts of subsidies on innovation activities in Germany. Scottish Journal of Political Economy, v. 52, n. 4, p. 565-586, 2005.
  • HÜNERMUND, P.; CZARNITZKI, D. Innovation Policy. ifo DICE Repor, v. 17, p. 37-54, 2019a.
  • HÜNERMUND, P.; CZARNITZKI, D. Estimating the causal effect of R&D subsidies in a pan-European program. Research Policy, v. 48, n. 1, p. 115-124, 2019b.
  • KANG, K.N.; PARK, H. Influence of government R&D support and inter-firm collaborations on innovation in Korean biotechnology SMEs. Technovation, v. 32, n. 1, p. 68-78, 2012.
  • KIM, Y.K.; LEE, K. Different impacts of scientific and technological knowledge on economic growth: Contrasting science and technology policy in East Asia and Latin America. Asian Economic Policy Review, v. 10, n. 1, p. 43-66, 2015.
  • KLETTE, T. J.; MØEN, J. R & D investment responses to R & D subsidies: A theoretical analysis and a microeconometric study. World Review of Science, Technology and Sustainable Development, v. 9, n. 2-4, p. 169-203, 2012.
  • KOBAYASHI, Y. Effect of R&D tax credits for SMEs in Japan: a microeconometric analysis focused on liquidity constraints. Small Business Economics, v. 42, n. 2, p. 311-327, 2014.
  • LACH, S. Do R&D subsidies stimulate or displace private R&D? Evidence from Israel. Journal of Industrial Economics, v. 50, n. 4, p. 369-390, 2002.
  • LANE, N. The New Empirics of Industrial Policy. Journal of Industry, Competition and Trade, n. 2010, p. 4041-4042, 2020.
  • LARÉDO, P.; KÖHLER, C.; RAMMER, C. The impact of fiscal incentives for R&D. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 18-52.
  • LE, T.; JAFFE, A.B. The impact of R&D subsidy on innovation: evidence from New Zealand firms. Economics of Innovation and New Technology, v. 26, n. 5, p. 429-452, 2017.
  • LEE, K. Economic catch-up and technological leapfrogging: The path to development and macroeconomic stability in Korea. Cheltenham: Edward Elgar Publishing Ltd., 2016.
  • LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.
  • LENIHAN, H.; HART, M. Evaluating the additionality of public sector assistance to irish firms: A question of ownership? Policy Studies, v. 27, n. 2, p. 115-133, 2006.
  • LOKSHIN, B.; MOHNEN, P. How effective are level-based R&D tax credits? Evidence from the Netherlands. Applied Economics, v. 44, n. 12, p. 1527-1538, 2012.
  • MAZZOLENI, R.; NELSON, R.R. Public research institutions and economic catch-up. Research Policy, v. 36, n. 10, p. 1512-1528, 2007.
  • MCKENZIE, K.J.; SERSHUN, N. Taxation and R&D: An Investigation of the Push and Pull Effects. Canadian Public Policy, v. 36, n. 3, p. 307-324, 2010.
  • MULKAY, B.; MAIRESSE, J. The R&D tax credit in france: Assessment and ex ante evaluation of the 2008 reform. Oxford Economic Papers, v. 65, n. 3, p. 746-766, 2013.
  • PARKHURST, J.O. The politics of evidence: from evidence-based policy to the good governance of evidence. Abingdon; New York: Routledge, 2017.
  • PAUNOV, C. The global crisis and firms’ investments in innovation. Research Policy, v. 41, n. 1, p. 24-35, 2012.
  • ROCHA, F. Does governmental support to innovation have positive effect on R&D investments? Evidence from Brazil. Revista Brasileira de Inovação, v. 14, p. 37-60, 2015.
  • RODRIK, D. Goodbye Washington Consensus, Hello Washington Confusion? A review of the World Bank’s Economic Growth in the 1990s: Learning from a Decade of Reform. Journal of Economic Literature, v. 44, n. 4, p. 973-987, 2006.
  • RODRIK, D. Why We Learn Nothing from Regressing Economic Growth on Policies. Seoul Journal of Economics, v. 25, n. 2, p. 137-151, 2012.
  • SEGARRA-BLASCO, A.; ARAUZO-CAROD, J. M. Sources of innovation and industry-university interaction: Evidence from Spanish firms. Research Policy, v. 37, n. 8, p. 1283-1295, 2008.
  • SUZIGAN, W.; GARCIA, R.; FEITOSA, P.H. A. Institutions and industrial policy in Brazil after two decades: have we built the needed institutions? Economics of Innovation and New Technology, v. 29, n. 7, p. 799-813, 2020.
  • SZCZYGIELSKI, K. et al. Does government support for private innovation matter? Firm-level evidence from two catching-up countries. Research Policy, v. 46, n. 1, p. 219-237, 2017.
  • WATANABE, C.; KISHIOKA, M.; NAGAMATSU, A. Effect and limit of the government role in spurring technology spillover - A case of R&D consortia by the Japanese government. Technovation, v. 24, n. 5, p. 403-420, 2004.
  • WESTMORE, B. R&D, Patenting and Growth: The Role of Public Policy. OECD, 2013. (OECD Economics Department Working Papers, n. 1047).
  • ZÚÑIGA-VICENTE, J.Á. et al. Assessing the effect of public subsidies on firm R&D investment: A survey. Journal of Economic Surveys, v. 28, n. 1, p. 36-67, 2014.

  • Funding:

    the authors declare that there is no funding

  • 1
    This article uses “evaluation” in a broad sense, including commissioned studies to conclude on the effectiveness of policies and academic investigations on intervention-specific issues.
  • 2
    Korea is an illustrative example shifting its emphasis from tariffs to R&D subsidies, in the late 1980s (LEE, 2019LEE, K. The Art of Economic Catch-Up. Cambridge: Cambridge University Press, 2019.).
  • 3
    For the sake of objectivity, many auxiliary interventions are absent, for example, the demand-oriented policies.
  • 4
    Among OECD members, R&D tax incentives is adopted by more than 20 countries.
  • 5
    Similar conclusions are observed in recent literature reviews (BECKER, 2015BECKER, B. Public R&D Policies and Private R&D Investment: A Survey of the Empirical Evidence. Journal of Economic Surveys, v. 29, n. 5, p. 917-942, 2015.; LARÉDO; KÖHLER; RAMMER, 2016LARÉDO, P.; KÖHLER, C.; RAMMER, C. The impact of fiscal incentives for R&D. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 18-52.).
  • 6
    Despite the scant evaluation of output additionality, the topic has received increasing attention from the literature in the recent period (LARÉDO; KÖHLER; RAMMER, 2016LARÉDO, P.; KÖHLER, C.; RAMMER, C. The impact of fiscal incentives for R&D. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 18-52.).
  • 7
    For an expanded overview of this evidence, see Cunningham et al. (2016CUNNINGHAM, P. et al. The innovation policy mix. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 505-542.).
  • 8
    In the same way that “institutional failures” can precede or reinforce the existence of “system failures”, which would require policies to correct this shortcoming.
  • 9
    An example is the methods to measure and compare costs and benefits over time. The current toolkit makes it difficult to capture both sides of the cost-benefit ledger since many of the benefits are based on future hopes for benefits and are fully captured only in the long run.
  • 10
    This aspect is notably relevant for most developing countries that tend to suffer more severely from economic downturns driven by local and international issues and obstacles to building coalitions around common goals given the uneven social structure.
  • 11
    See Rodrik (2012RODRIK, D. Why We Learn Nothing from Regressing Economic Growth on Policies. Seoul Journal of Economics, v. 25, n. 2, p. 137-151, 2012.) for a detail discussion of these evaluation failures in the industrial policies context.
  • 12
    For discussion of sequencing policy interventions in the case of energy agencies, see Cunningham et al. (2016CUNNINGHAM, P. et al. The innovation policy mix. In: EDLER, J.; CUNNINGHAM, P.; GÖK, A.; SHAPIRA, P. Handbook of Innovation Policy Impact. Cheltenham: Edward Elgar Publishing, 2016. p. 505-542.).
  • 13
    In addition, while the analysis of the results is sufficient to assess the “efficiency” of these policies, the “effectiveness” of the catching-up process involves some structural changes that may require even more complex framework and instruments.

Publication Dates

  • Publication in this collection
    15 Nov 2021
  • Date of issue
    2021

History

  • Received
    23 Sept 2020
  • Reviewed
    31 Mar 2021
  • Accepted
    07 July 2021
Universidade Estadual de Campinas Rua: Carlos Gomes, 250. Bairro Cidade Universitária, Cep: 13083-855 , Campinas - SP / Brasil , Tel: +55 (19) 3521-5176 - Campinas - SP - Brazil
E-mail: rbi@unicamp.br
Accessibility / Report Error