Building up a national network of applied R&D institutes in an emerging innovation system

Abstract

Paper aims

This paper aims at answering the research question “How to successfully build up and strategically manage a new network of applied R&D in Brazil?”

Originality

The paper is based on a unique experience of a strategic partnership, transferring the experiences of managing the largest network of applied research in Europe to the Brazilian National Innovation System (NIS).

Research method

The research described in this paper follows an action research approach, using a participative process of rapid prototyping, pilot tests and continuous revision and adaptation.

Main findings

The paper presents a comprehensive and consistent set of management models, procedures and tools for the planning, implementation and evaluation of applied R&D institutes.

Implications for theory and practice

The paper’s findings contribute to the empirical research on methodologies to manage knowledge-based networks and innovation actors at the interface between research and industry.

Keywords:
Applied research; National innovation system; Brazil; Network governance; Business planning; Strategic Management; Evaluation

1. Introduction

With a GDP of 2.138 billion USD (2018), Brazil is the ninth largest economy of the world. SENAI is the National Service for Industrial Apprenticeship and belongs to the National Confederation of the Industry (CNI) in Brazil. Its main mission is to provide technical education to qualify the industrial workforce of Brazilian companies. Despite previous experiences in providing technological services, such as metrology and technical consulting, the business area of applied research, technological development and innovation (RDI) was almost entirely new to the organization with approximately 20,000 staff in various operational units distributed over whole Brazil.

Fraunhofer Gesellschaft is the largest organization for applied research in Europe with over 27,000 staff in more than 75 institutes distributed over Germany, and with various international partnerships and subsidiaries. The Berlin-based Fraunhofer Institute for Production Systems and Design Technology (IPK) has vast experiences in international consulting regarding the development of regional and national innovation systems. IPK’s division Corporate Management is specialized in developing and implementing management systems for companies, public clients and research institutes. Based on these experiences, SENAI assigned Fraunhofer IPK with supporting the establishment of the new national network of applied R&D institutes in Brazil.

In 2012, when the project to build up the network of 25 SENAI Innovation Institutes was started, Brazil was ranking on the 58^th position of the Global Innovation Index (Mobilização Empresarial pela Inovação, 2018a) and the economic scenario in the country was characterized by concluding a decade of strong economic growth, which was partly based on the increasing exports of raw material and commodities, such as meat, coffee, soy and oil, among others. The other part of the economic growth was mainly triggered by internal social programs which increased the buying power of the Brazilian population significantly, and thus, strengthened the domestic market for consumer goods on a broad basis, taking the population of approx. 200 million inhabitants and Brazil’s continental size into account.

Already foreseeing at that time, that this economic scenario would not serve for sustainable economic growth and future increase of national wealth in the long run, the leading industrial players and large Brazilian companies articulated the need for a shift towards higher added value in the national production chains, including the increase of productivity and added value through technology and innovation. This led the Entrepreneurial Movement for Innovation (MEI), consisting of the CEOs of the largest industrial companies in Brazil (Mobilização Empresarial pela Inovação, 2018b), to request a national initiative with the aim to support the Brazilian industry in tackling this challenge of introducing technology and innovation to the companies as a means to strengthen the competitiveness of the Brazilian industry in a globalized economy.

Seven years later, after a severe economic crisis and dramatic political turbulences, and Brazil ranking on the 64^th position of the Global Innovation Index (Mobilização Empresarial pela Inovação, 2018a), this need becomes ever more evident. Recent developments, like the creation of a national funding program for industrial research and innovation (EMBRAPII), the free trade agreement between MERCOSUR and the European Union as well as attempts to reduce bureaucracy and the protection of the domestic market, have pointed into a favorable direction. At the same time, the constraints and barriers for industry-financed R&D remain high in the current Brazilian economic scenario with a history and business culture not yet acquainted with investments in technological innovation on a large scale. Breaking up these barriers at least partly and demonstrating the economic benefits and return of investment of industrial R&D, is thus, one of the market challenges this new applied R&D network has to face.

2. Background and methodology

Innovation is the driving factor for economic development, growth and the wealth of nations (Schumpeter, 1912) and is widely understood as a complex process (Drucker, 1985) involving different types of actors from the public and private sector (Mowery & Rosenberg, 1993; Chesbrough, 2003; Hauschildt et al., 2016), often organized in networks (Kozioł-Nadolna & Świadek, 2010; Barbieri & Álvares 2016; Taferner, 2017). These actors from the different societal sub-systems together form the National Innovation System (NIS), a term first introduced by Freeman (1987) and defined as: “[...] the network of institutions in the public and private sectors whose activities and interactions initiate, import, modify and diffuse new technologies.” (Freeman, 1987). Nowadays, these inter-organizational networks are viewed as a major driver of innovation (Ozman, 2009; Ringwelski, 2017), which has led to a sharp increase in research and publications on innovation networks in the last two decades (Battista Dagnino et al., 2015).

On the macro-level, successful innovation processes rely on the effective interplay between different organizations from science, industry and government which is usually referred to as the “triple helix” concept (Leydesdorff & Etzkowitz 1995). Intermediaries are key to overcome structural challenges inside these innovation networks and are defined as hybrid organizations which operate at the interface between two or more sub-systems of the triple-helix model (Ranga & Etzkowitz 2013), e.g. technology transfer centers, venture capital firms, business angel networks or Research and Technology Organizations (RTOs). RTOs link research and private sector innovation with the task of transferring scientific results to the private sector (Organisation for Economic Co-operation and Development, 2011). Examples of RTOs are the Fraunhofer Society in Germany, TNO in the Netherlands, VTT in Finland, Tecnalia in Spain and SINTEC in Norway (Organisation for Economic Co-operation and Development, 2011).

The role and functions of RTOs in Innovation Systems has been investigated in comprehensive research (Organisation for Economic Co-operation and Development, 2002; Roll-Hansen, 2009; Müller-Prothmann & Dörr, 2014), including a recent benchmarking study by MIT with a focus on the financial model of RTOs worldwide, examining the distribution of public funds and private (industrial) revenue (Reynolds et al., 2019; Zylberberg, 2017), as this mix of income is viewed as one of the specific operational characteristics of RTOs. The European Association of Research and Technology Organisations (EARTO) defines the function of an RTO as an organization which predominantly offers R&D, technological and innovation services to enterprises, governments and other clients (European Research Advisory Board, 2005). The majority of the investigated RTOs focuses on applied research and experimental development rather than basic research (Zylberberg, 2017), as their mission to transfer technology to industry requires application-oriented research results.

In this context, the 25 SENAI Innovation Institutes, being the research object of the present article, shall be classified as RTOs as defined above. Taking into account the existing examples and experiences with national networks of RTOs, SENAI decided to define a transversal technology and research field as the scope of actuation for each Innovation Institute, to be distributed over Brazil. The technology and research fields were chosen based on the current and future demand for technological solutions to increase the competitiveness of the Brazilian industry and its main sectors. The geographical distribution of the 25 Innovation Institutes was based on multiple criteria, such as experiences with certain technologies in existing operational units of SENAI, the proximity to clusters of potential industrial clients in a certain federal state or geographical area, among further technical and political criteria to ensure support and commitment of the main stakeholders. Figure 1 shows the fields of actuation and the geographical distribution of the 25 SENAI Innovation Institutes:

Figure 1
National Network of 25 Operational SENAI Innovation Institutes in Brazil.

Taking this pre-defined scope of the national RTO network as the initial situation, the present paper focuses on the practical implications of the ambitious endeavor to implement such a national network of Innovation Institutes from scratch and to direct it towards successful operations. Thus, the main research question of this analysis is defined as:

How to successfully build up and strategically manage a new network of applied R&D institutes with the aim to strengthen the industry’s competitiveness in Brazil?

The management approach to solve this practical challenge had to cover two levels as a minimum prerequisite: On the network level overall strategies, objectives and guiding principles had to be defined and transferred into national standards for quality assurance and successful operations of all SENAI Innovation Institutes. On the micro- or actor-level, i.e. the level of the single institute, the main challenge was to develop and deploy adequate management models, methods, procedures and tools to support a systematic planning, implementation and continuous evaluation of each Innovation Institute in the light of the strategies and principles defined on the network level.

Effective and efficient network management requires actionable methods. Sydow (2010) points out that, despite the considerable variety of research on networks, much is still unknown about practical network management. Becker et al. (2011) agree that literature dealing with the phenomenon of networks under practical considerations is still scarce, underscoring the notion that the transfer of traditional management practices to the network context is at present inadequate. Despite the available practical experiences from existing RTO networks and their management systems, there are no widely accepted standards for comprehensive management systems and methodologies for RTOs. As many different types of operational models of RTOs exist in practice (Zylberberg, 2017), a standardization approach might also not be adequate. Therefore, general approaches and methods of strategic management as well as practical experiences with the management of international RTO networks were taken as a basis to develop a practical set of management methods and tools for the planning, implementation and evaluation of the SENAI Innovation Institutes, adapting the existing methods to the specific Brazilian requirements and environment.

Furthermore, the management methods and tools to be developed had to fulfill the specific requirements of an Innovation Institute: First, the nature of an RTO as a knowledge-intensive business needs to be taken into account, integrating the main assets of applied R&D actors into a comprehensive management model, i.e. the intangible resources and strategic success factors of each institute need to be displayed and turned into measurable, and thus, manageable objects (Will, 2012). Second, the two generic approaches of the innovation process need to be incorporated in an adequate management method for RTOs as an innovation intermediary between science and industry: the “market-pull” as well as the “technology-push” approach (Corsten et al., 2006; Müller-Prothmann & Dörr 2014). This corresponds to the third methodological requirement, reflecting a discussion with a long history in strategic management research: the management system needs to integrate the resource-based (Barney, 1991) and the market-based view (Porter, 1996) of the organization.

The deployed methodology to solve the research question stated above is following an action research approach from the point of view of the involved project managers of the two main organizations responsible for the development of an adequate management system for the network of Innovation Institutes in Brazil, SENAI and Fraunhofer IPK. Due to its characteristics, action research was chosen as the most suitable way to achieve a compromise between a structured research process and applicable results (Tripp, 2005). Because of these characteristics, action research can construct a suitable framework for the application of the developed method and its iterative improvement under genuine conditions (Coughlan & Coghlan, 2002; Mertler, 2017).

Following this basic action research approach, using existing standards in strategic management as well as practical experiences with managing existing national RTO networks as a first basis, prototypical models, methods and tools were developed in an agile manner, then tested in pilot applications and subsequently adapted and improved to serve the reality of the SENAI Innovation Institutes in their specific environment. In a participative approach these models, methods and tools were being reviewed according to the expected and produced outcomes by the user community in regular project meetings, i.e. by the directors and senior researchers of the Innovation Institutes on the actor-level as well as by the national department of SENAI as the central unit responsible for the coordination of the institutes on the network level. The results of these regular reviews were being integrated into the next cycle of development and improvement, and thus, leading to stable versions of management modules according to the specific needs of the institutes in each lifecycle phase. In synchrony with the process of growing and maturing the network of institutes, further modules with more elaborated features were developed and added to the final management system.

3. Shaping a network strategy for 25 applied R&D institutes

In the context described above, a national initiative for innovation was being designed, gathering major political supporters like the Ministries for Science and Technology (MCTIC) and for Industry and Foreign Trade Development (MDIC), including a strong engagement of the Brazilian Development Bank (BNDES). Triggered by the industry via MEI and other industrial channels, CNI took the lead in this initiative and assigned the challenging task to implement a national network of 25 “Innovation Institutes” to the national department of SENAI, integrating the Industry Federations in 13 states of Brazil and the respective regional departments of SENAI in these states as operational leaders for the actual physical and technological execution of this initiative.

In a first phase from 2012 to 2018, the 25 SENAI Innovation Institutes (ISIs) elaborated adequate business plans to subsequently steer the implementation of each institute in terms of scientific-technological infrastructure and qualified research team as well as to start the operations by executing first R&D projects for industrial clients. After this initial implementation and “ramp-up” phase had been concluded successfully with the 25 ISIs being operational, the focus of the responsible national department of SENAI and of the newly installed “ISI Network Governance Committee” shifted towards the strategic development and positioning of this new applied R&D network in the National Innovation System (NIS) of Brazil. Taking past experiences of the Fraunhofer Gesellschaft in Germany and a recently developed typology of Research and Technology Organizations (RTOs) by MIT (Reynolds et al., 2019; Zylberberg, 2017) into consideration, the following model for the mid- to long-term development and strategic positioning of SENAI’s innovation business was created (see Figure 2):

Figure 2
Strategic Positioning of SENAI’s Innovation Network in National Innovation System (NIS).

In contrast to the usual development of public universities and research institutions towards becoming a professional applied R&D provider, which usually starts from the role “creator” (based on their public mission to create new knowledge and qualifying people), the SENAI Innovation Institutes did not start from this publicly funded scientific basis. With almost no scientific track record and no public basic funding for research, but being integral part of the industry federation, the strategy adopted by SENAI was to start positioning the ISIs behaving as “contractors”, i.e. building up the capabilities and reputation to perform high quality contract research with a major part of the revenues coming directly from the industry. This strategy requires a clear demand-oriented market-pull approach with a focus on a professional industry-compatible culture of working and delivering the respective technological solutions.

Once having reached a certain maturity and reputation in delivering high quality research results with immediate practical benefit to the industry, a possible and natural next strategic stage to be conquered by the ISI network is the “convener” role, i.e. striving for and practicing the behavior of a “hub” or “netweaver” which attracts and integrates various different players from the research and the industrial world to format and steer larger and more strategic R&D programs, such as project consortia or innovation clusters with a long-term and/or disruptive research agenda, including different R&D partners and industry associations, e.g. focusing on the technological transformation of a whole industrial sector.

Once the position on the national market as a trustful and professional R&D hub is reached, a major next strategic stage focuses on the systemic layer of the national innovation system, using the strong reputation of the ISI network to influence innovation-related policies on the national level, i.e. becoming system-relevant for the NIS and thus, supporting to shape the national long-term R&D strategies as an important intermediary within the “triple-helix” approach, aligning the macro research agenda with the industry’s agenda and the federal government to define synergetic industrial and research policies. Eventually, this strategy should also lead to sustain a more comprehensive “creator” role, also including a joint basic research and “technology push” agenda with partner universities and research centers, necessary for certain industrial innovation results in the long-term and the respective formation of new professionals in emerging fields of technology.

4. Strategic network governance and lifecycle management

In order to operationalize this strategy, a national network governance structure and a respective lifecycle management for the 25 SENAI Innovation Institutes had to be defined and implemented. On a macro-level this process of governing this R&D network was defined as follows (see Figure 3):

Figure 3
ISI Lifecycle Management as the Core ISI Network Governance.

The “Guiding Principles” were derived from the normative requirements of the main stakeholders and served as the starting point for elaborating the strategies and objectives to be operationalized and supported by the network lifecycle management:

• Industry Orientation: Each ISI shall predominantly work for the benefit of the industry, providing innovative technological solutions to increase the competitiveness of the industry in Brazil.

• National Actuation: Different from all other units inside the federative organization of SENAI, each ISI shall operate on a national level, offering and providing solutions in a nationally agreed transversal research or technology field.

• Excellent Applied Research: Each ISI shall strive for excellence in applied research and technological development, delivering innovative solutions with clear benefits to the industry and society in Brazil, performing on a state-of-the-art level of R&D.

• Internal Collaboration: Conceptualized as a synergetic network from the beginning, each element of the network (the ISI) shall seek collaboration on a resource, market and technology level with its counterparts in the network, creating a strong network value proposition together on the market, respecting the boundaries of its own research and technology area as defined in the network’s national R&D portfolio to avoid significant technological overlaps and resulting competition inside the ISI network.

• External Cooperation: Following the concept of applied research, each ISI shall act as an intermediary player between basic research and industrial application, and thus, seek strategic cooperation with external R&D partners (e.g. universities, national and international R&D institutes), creating win-win-situations based on a complementary profile of actuation.

Further basic requirements of the main stakeholder and mother-organization SENAI included the following business objectives:

• Financial self-sustainability: Each ISI shall be able to cover its own operational costs by its own revenues by the end of the ramp-up phase, but at latest in the 8^th year of operation.

• Focus on applied R&D and innovation: Due to parallel structures at SENAI dedicated to basic or off-the-shelf technological services (e.g. metrology and consulting on mature technologies), the so-called SENAI Technology Institutes (IST), it is important to ensure that each ISI focuses on its main business purpose “innovation”, i.e. developing new technological solutions applicable in the Brazilian industry, mainly operating in R&D projects, rather than selling and applying ready-made technological services.

Taking these guiding principles and stakeholder requirements as the initial point of departure to design an adequate management system in order to support effective network governance, three main phases of the lifecycle of a typical SENAI Innovation Institute were defined. Each lifecycle phase displays a particular stage of maturity in the evolution of each institute. Considering that a) most of the first 25 institutes were built up from scratch, b) most of the involved regional departments of SENAI had little to no experience with professional research and development, and that c) the actuation in the field of technological innovation represented a completely new business area for SENAI at the national level, it may well be valid to use the three main human maturity stages as an analogy for the three corresponding lifecycle phases: from “child” to “teenager” to “adult”. The following chapters are dedicated to describing these three lifecycle stages and the respective management system, methodologies and tools which were developed and used to support effective governance on the network level in each phase.

After the initial planning and ramp-up phase, the to-be-developed management system should continuously support the effective governance of the ISI network as a whole. For this purpose, a Network Governance Committee was installed, composed by representatives from SENAI national department, from the involved regional departments and from the Innovation Institutes as well. In regular meetings this committee continuously monitors and controls the evolution of maturity of each institute and takes the relevant decisions to maintain all operational ISIs on a high quality level by fulfilling the national minimum requirements. In case of significant deviations from these minimum requirements, the respective ISI would be set on the status “under observation” (yellow light) and the supporting governance structures would help the ISI to manage the turnaround back to “normal operations” (green light) by the means of coaching and mentoring activities up to comprehensive “rescue programs”, if necessary. In the rare case of continuing violation of the network’s minimum quality requirements the ISI would be put on the “Exit / Transformation” status (red light) and the Network Governance Committee together with the responsible state’s industry federation would decide on measures to withdraw the status of a full-scale Innovation Institute, re-define the scope of the institute and/or re-allocate parts of the respective institute to other entities in the network, assuring that the nationally agreed R&D portfolio continues to be covered by the ISI network. In each phase, the respective lifecycle management system needs to carefully define and monitor relevant criteria to measure the fulfillment of minimum requirements in an objective way in order to support these governance processes systematically.

4.1. Entering the network

The first 25 SENAI Innovation Institutes were led through the initial phase by a structured process of Strategic Business Planning. Based on the methodology “Integrated Strategy Development” (Will, 2012; 2020), a workshop-based approach was chosen in order to stimulate the participation of the regional responsible units and the core team of the respective institute from the beginning of the planning and implementation process.

Following the overall strategy of the ISI network (see Figure 2), the standardized process for Strategic Business Planning shown in Figure 4 is clearly focusing on a demand-oriented market-pull approach from the start. The chosen method, originally designed to structure strategy processes in small and medium-sized companies, supports this business-focused view of the institute with the aim to concentrate the initial planning activities on a “contractor” behavior, i.e. prioritizing the successful actuation as a professional R&D provider for the industry.

Figure 4
ISI Strategic Business Planning Procedure.

After an initial phase of preparation and market analysis, a sequence of workshops moderated by external experts was executed to develop a strategic business plan step-by-step. As a first crucial step to define its market-driven strategy, the institute’s strategy planning team elaborated and evaluated the main market segments to be targeted. Using the data from the prior quantitative and qualitative market analysis, the industrial sectors with relevant demand were identified and defined by describing their demand for R&D solutions and listing existing and potential customers in the respective sector.

Using an adaptation of the original BCG-Matrix (Henderson, 1970; Will & Wuscher, 2014), these main market segments were then assessed by the strategy planning team on a 10-point-scale in two dimensions: “Market Attractiveness” (demand for R&D solutions in the ISI’s research and technology field) and “Probability of Market Entry” (based on existing relations to the sector and general entry barriers of the sector). The resulting Market Attractiveness Portfolio (see Figure 5) serves as a decision basis for a first prioritization of markets to be targeted pro-actively by the institute.

Figure 5
Exemplary Market Attractiveness Portfolio.

As a second step, the R&D offerings of the institute were structured into Main Service Areas, bundling potential R&D activities in market-oriented packages. Bearing the qualified demand of the prioritized market segments from the first step in mind, the institute’s strategy planning team discussed how to structure its service offer for those industrial clients in order to be attractive to them. In a similar methodological approach as in the first step, the Main Service Areas were then assessed by the strategy planning team in two dimensions: “Product / Service Competitiveness” (market demand for own services in relation to potential competitors) and “Product / Service Readiness” (adequacy of institute’s competence base and infrastructure for delivering services in respective service area). The resulting Product Attractiveness Portfolio was then used to prioritize those Main Service Areas with high readiness and competitiveness for actual R&D projects to be acquired and executed as first operational activities in the ramp-up phase (see Figure 6).

Figure 6
Exemplary Product Attractiveness Portfolio.

While this initial business planning phase for the first 25 SENAI Innovation Institutes used a guided process and a rather prescriptive approach in order to stimulate the necessary mind-setting for this market-oriented applied research model, future institutes will be granted the label “ISI” based on technical criteria and a strategic decision by the network’s Governance Committee in order ensure the minimum requirements described above as entry criteria for the network. These requirements include the demonstration of clear and evident industry demand for a certain technology field and a pre-defined service portfolio that complements the national R&D portfolio of the network in a strategic manner. A respective business plan has to be verified by a technical team and be approved by the ISI Network Governance Committee.

4.2. Ramp-up phase

After the successful termination of the initial planning phase, resulting in a business plan agreed by the internal and external stakeholders and investors, each SENAI Innovation Institute was approved to start the implementation process and the first operational activities in parallel. By definition, the “ramp-up” phase is characterized by these two parallel processes and the careful and strategic alignment between these two highly interdependent activities, which was the first practical challenge of the responsible ISI managers. Consequently, a maturity model was defined that puts the focus on exactly this challenge, closely monitoring the evolution of the implementation of the planned infrastructure, service areas and research team as laid out in the business plan on the one hand, and the success in acquiring and executing first research projects on the market, on the other hand. To master this complex task, it was crucial to e.g. align the acquisition of certain equipment and competencies in accordance with the research area which showed the highest readiness (and competitiveness) in the initial business planning, transferring this plan into concrete activities to acquire first projects in this prioritized business area. This is not only important to achieve a first market-entry as fast as possible, generating valuable experience and reputation on the market without waiting for all service areas to be fully set up, but also to use these first experiences and feedback from the market to revise certain parts of the business plan, continuously adapting the strategic development of each institute to the reality on the market. In this way, it was possible to optimize the purchase of costly machinery and equipment as well as the attraction and contracting of adequate research staff according to the real needs of the market.

To track and support this maturity evolution of each institute, a basic maturity model was defined, summarizing the objectives of maturity evolution from planning to implementation, stabilization and excellence phase (see Figure 7). Based on this basic concept, a more specific maturity model was derived to determine the specific stage gates (milestones) of maturity evolution for the particular case of the SENAI Innovation Institutes. Each stage gate is characterized and operationalized by a set of criteria and performance targets which need to be achieved in order to advance in the respective maturity levels of the model. In this manner, a “guided” evolutionary process could be supported in a systematized way which was particularly important for the ramp-up phase, in which most of the ISIs were starting their operational activities from scratch, i.e. without previous experience in applied R&D for the industry. In this phase, a certain “prescriptive” approach was needed to speed up the implementation process and the respective learning curve by providing managerial support according to the specific needs of each maturity level (see Figure 8). The “Maturity Check” operationalized these minimum requirements by a list of criteria for each maturity milestone, and respective evidences being stored in an online repository to allow the remote analysis of the maturity criteria (Kohl et al., 2016).

Figure 7
Maturity Concept Model for Applied R&D Institutes.

Figure 8
Maturity Stage-Gates Along the ISI Lifecycle.

To support the systematic evolution along the maturity stage gates of the ISI lifecycle, several management tools, techniques and procedures were used and implemented to analyze the internal resource base, to structure the internal core processes and to derive strategic action plans to close the gaps between the defined future status, laid out in the strategic business plans (regularly being updated), and the actual status of maturity of each institute. Among these tools and techniques, the well-proven assessment of intellectual capital regarding the three dimensions quantity, quality and systematic management (QQS Assessment) was used in a structured workshop procedure with a representative team of the respective institute, moderated by experienced external experts (European Commission, 2008; Alwert et al., 2008). The resulting QQS Portfolio, summarizing the assessed actual status of nine success factors for applied research institutes (Kuhlmann & Holland, 1995) against the target status, was then used as a discussion basis to derive the most urgent and important actions to close the strategic gaps in the resource base to advance the implementation of each institute systematically and as efficiently as possible (see Figure 9). Special attention and priority for action was put on those success factors in human capital (HC), structural capital (SC) and relational capital (RC) which showed a relatively low actual status in the quantity and/or quality dimension (X- and Y-axis of the portfolio) as well as a relatively low value in systematic management (size of the bubble in the portfolio), as those factors show the highest improvement potential according to the strategic objectives of each individual institute.

Figure 9
Self Assessment of Institute’s Intellectual Capital (QQS Assessment).

Based on the documentation of the structured discussions in the assessment workshop, the main challenges and gaps in those prioritized success factors could be easily retrieved and analyzed in detail to derive the right strategic actions accordingly. This task was performed by the same representative team of each institute in a next workshop session, moderated by external experts. After an initial brainstorming on potential actions to close those prioritized gaps, the collected suggestions were assessed again according to the dimensions Importance, Urgency and Simplicity on a simple 3-point scale, resulting in a ranking to prioritize those actions with the highest impact and relatively low complexity and effort in order to ensure quick wins in this initial ramp-up phase. The resulting action plan was finally transferred to the Implementation Roadmap, displaying the most important actions on a timeline for one year (see Figure 10).

Figure 10
Strategic Action Plan for Systematic Development of Institute’s Resource Base (Example).

Procedures for fine planning and monitoring of strategic actions (Alwert & Will, 2014) were introduced to manage the implementation of the strategic actions systematically, generating transparency on the progress of the institute’s development and, thus, a basis for regular follow-up assessments by the institute itself and by the responsible regional department in partnership with the national department of SENAI, responsible for coordinating the whole implementation process on the network level.

Based on these initial steps of implementation support, a comprehensive evaluation system was developed, taking into account the maturity levels mentioned above, combining basic organizational requirements, such as a minimum team of researchers and a respective lab infrastructure, with performance targets according to a first set of Key Performance Indicators (KPI), measuring the institute’s initial operations, such as first projects contracted and respective revenues.

While this approach and logic of continuous evaluation of the institute’s performance and maturity evolution was introduced and established in the ramp-up phase of the ISI network, the full evaluation system was then developed to serve the needs of the next phase of the ISI lifecycle, i.e. the Stable Operations Phase, using a standardized KPI system to operationalize the overall strategic principles and objectives of the ISI network, as introduced above.

4.3. Stable operations phase

After successfully concluding the Ramp-up Phase, which by definition is characterized by the parallel challenges of implementation and initial operation, each SENAI Innovation Institute had to fulfill a set of minimum requirements to formally enter into the Stable Operations Phase. As a “full-grown” institute, the ISIs had to prove that all research and service areas outlined in the business plan had been installed and were operational, as well as to show a certain minimum size of operations by the total revenue and a certain rate of cost coverage by own revenue, among other criteria.

Entering this final stage of maturity evolution, the SENAI Innovation Institutes were now directed to put their focus on a continuous strategic management cycle in order to continuously improve their performance and systematic growth of each institute. For this purpose, and based on the model of Integrated Strategy Development (Will, 2012; 2020) which was already used during the initial business planning phase, a standardized framework for a strategic business model was created which each institute had to fill with individual content to systematize its own business strategy. This framework and standard model then also served as a basis for a comprehensive and integrated evaluation system in the next step of the elaboration of the governance and management system for the whole ISI network.

While the level of the Business Areas, i.e. the R&D products and services and focus market segments of each institute, already addressed in the initial business planning phase, focuses the business question “what are we selling to whom?”, the Value Creation Model, as the second part of the strategic business model, aims at answering the question “How are we going to produce the value for the customer and how do we achieve our desired business results?”. Based on the framework of the Integrated Strategy Development, and in line with standard management models like Total Quality Management (Zink, 2004; European Foundation for Quality Management, 2010) or the Balanced Scorecard (Kaplan & Norton, 1996) and the respective Strategy Maps (Kaplan & Norton, 2004), the Value Creation Model follows the logic of (1) Resources being utilized in (2) Business Processes, to create (3) Customer Value and (4) Business Success. Taking into account the importance of intangible resources in the case of the knowledge-intensive Innovation Institutes, special focus was given to the Intellectual Capital and the respective success factors for applied research institutions in the “Resources” dimension.

The Value Creation Model itself was developed together with the first SENAI Innovation Institutes in a moderated workshop procedure, discussing the main elements of successful operations. Starting from the right side of the model, i.e. from the business results to be achieved, the leading questions were:

Business Success

• Which overall results do we have to achieve in the mid- and long-term perspective in order to fulfill our mission/vision?

• Which impact do we want to achieve externally?

• How do we measure our overall success of operations?

Customer Value

• What do our customers value regarding our services?

• What are/will be our main competitive advantages? How do/will we differentiate ourselves from competitors?

• Which position on the market do we want to achieve?

Business Processes (Value Adding Core Processes)

• Which core processes do we need in order to achieve the defined Business Success?

• How do we acquire projects and generate revenue?

• How do we produce the planned products & services?

• How do we generate the defined value for our customers?

• What are the operational objectives of our Business Processes?

• How do we measure that the objectives are achieved?

Resources / Intellectual Capital

• Which resources do we need in order to drive our business processes effectively?

• Which success factors are crucial to achieve our strategic objectives and to produce the desired customer value and business success?

• Which knowledge, competencies, structures and relations do we need to be successful?

• Which are our (intangible) assets that differentiate ourselves from the competition?

The overall strategy for the ISI network, its guiding principles and strategic objectives served as a high level starting point for discussing the questions above and thus, defining the main elements of the Value Creation Model and the individual objectives of each element for every Innovation Institute. The outcomes of these discussions with the individual Innovation Institutes were harmonized and aggregated into the following standard structure of the general Value Creation Model of the ISI Network (see Figure 11):

Figure 11
ISI Value Creation Model as a Basis for Strategic Management and Evaluation.

This standardized Value Creation Model now served as the basis for the planning of the strategic actions to implement and continuously develop each institute according to the overall objectives of the network. In parallel, it served as the standard structure for the continuous assessment and evaluation of its operations, and thus, as the basis for the development of a comprehensive management evaluation system. Following the maturity model described in the previous chapter, the full evaluation system was now enhanced by a performance monitoring system, using a set of standardized indicators according to the structure of the Value Creation Model.

SENAI’s national department then assigned Fraunhofer IPK with executing so-called Management Audits to continuously assess the performance of each institute on-site and to derive a strategic action-map according to the individual gaps and improvement potential to be covered. This one-day workshop was designed as a combination of an audit procedure, checking the adherence of certain minimum requirements according to the maturity model, and a management coaching approach in which the ISI Director discussed the actual performance of his/her institute with a team of selected ISI staff and representatives of SENAI’s regional and national department, moderated and coached by external management experts of Fraunhofer IPK. The Management Audit, occurring on-site every two years as the default procedure, aimed at creating a management agreement between the institute, the regional and the national department. This management agreement is registered in a Management Audit Report which summarizes the findings of the actual status and performance of the institute as well as respective actions, investments and support needed to overcome weaknesses and to continue the development towards a full grown and stable R&D provider for the Brazilian industry.

Besides the full audit report with all details of the analysis and derived measures and actions, an executive summary of the audit’s findings was produced for each institute, summarizing the main corner stones of the individual strategy in three slides, using the same standard structure of the ISI Value Creation Model to facilitate reading and interpretation of SENAI’s management staff in a standardized manner and to ensure a consistent logic of the findings. This logic follows the basic approach of strategic management to 1) (re-)define long-term strategic objectives, i.e. the target status of the business model and its operational performance, 2) analyze and assess the current performance based on a set of appropriate standard indicators, i.e. investigate the actual status, and 3) derive strategic actions to close the gap between the actual status and the desired target status (see Figure 12).

Figure 12
Basic Logic of Management Audit – One Model, Three Consistent Tools.

As for the long-term strategic objectives, each of the pre-defined elements of the Value Creation Model had to be described qualitatively regarding the individual target status by each Innovation Institute. A generic version of this Strategic Objectives Map, valid for the whole ISI network, is shown in Figure 13.

Figure 13
Strategy Map with Long-Term Strategic Objectives.

Based on this initial definition of the desired future status, the next challenge was to find an adequate set of indicators to measure relevant attributes of the elements of the Value Creation Model in order to execute the quantitative performance analysis. In a first bottom-up approach, the Innovation Institutes were asked to gather possible KPIs for the main strategic aspects of their business model. This approach follows the principle that the success factors in the four pillars of the Value Creation Model may be operationalized and measured by individual indicators, i.e. defining customized KPIs for each specific strategy and case. Still, the model’s standard structure would provide a standardized general framework to allocate and interpret these individual indicators. But of course, the quantitative data and values would not necessarily be comparable among the different institutes. Therefore, to achieve the requirement of a unified governance of all institutes inside the national network, a set of standard KPIs was needed, leaving room and the possibility to add any specific indicators on the institute level later. The logical hierarchy used to operationalize the qualitative description of the Value Creation Model and its elements by quantitative indicators is shown in Figure 14. This hierarchy follows the assumption that indicators may, in the best case, measure relevant aspects of a certain strategy, i.e. a certain part of a business mode, on an objective basis, but that a few indicators can never represent the whole picture and the full complexity of a business or an innovation institute. Therefore, a higher level interpretation context is needed to focus the performance analysis on really relevant aspects and objectives, to be found in the pre-defined success factors and their interplay in the Value Creation Model, as part of the overall Business Model of the SENAI Innovation Institutes.

Figure 14
Logical Hierarchy of Levels of Performance Analysis.

In order to elaborate this standard set of indicators for the ISI network, a careful selection of indicators had to be conducted, solving the conflicts between validity and the practical measurement process and effort. When trying to measure, for example, the actual contribution to the Customer Value factor “Optimized Productivity”, several institutes suggested measuring the actual cost savings at the client companies achieved by implementing a certain technological solution, developed by the respective SENAI Innovation Institute, e.g. in the company’s production process. No doubt, this would be the “best” quantitative and objective measure to really know if and how much the Innovation Institutes were helping to improve the industry’s competitiveness – one of the highest strategic goals of the network. But in practice, this kind of data is very hard or impossible to acquire on a reliable basis, as it would rely on the customer’s own data which could either be inaccurate or subject to confidentiality, i.e. sensitive internal data that some companies would never disclose. Moreover, it is scientifically difficult to attribute certain causes to a specific effect in a non-controlled environment, i.e. in “real-life” practice where many external and internal factors influence the performance and the productivity of a specific production process (high complexity). In other words: even if a company shows a certain measurable increase in productivity (e.g. same output with reduced costs), it is not automatically proven that this was (only) caused by the introduction of a new technology. Many other causes could also have an effect on these reduced production costs, as for example lower raw material or energy prices, variations in the orders being produced by the respective production process, deviation in the down-time of machines etc. Adding the issue of time lags which many innovations show in terms of producing economic effects, it becomes a very challenging endeavor to try to measure this kind of accurate monetary contribution to a company’s productivity. Even if these challenges of data gathering and data interpretation could be overcome by a highly systematic and scientific measurement and analysis process, the necessary effort for this (secondary) measurement would, in many cases, exceed the added value of the respective R&D project, i.e. the actual technological work of the institutes for their clients.

That is the main reason why the set of standard indicators, serving the need of all SENAI Innovation Institutes, had to be somewhat pragmatic in the selection of feasible KPIs, striving for an optimum between validity and the effort for (re-)producing the data analysis. The chosen indicators are the result of a systematic investigation of standards and best practices in the field combined with a systematic discussion inside the national department of SENAI, taking into account the specific requirements and context of the ISI network as well as the individual suggestions from the institutes themselves. Special attention was given to the following criteria when choosing the Key Performance Indicators (KPI) for the SENAI Innovation Institutes:

• Relevance for strategic objectives and operational model of SENAI Innovation Institutes

• Low effort for data gathering (KPI already exists or data is available)

• Semantic link to the Value Creation Model as the interpretation context of the KPI values

• Compatible with other KPIs to perform multi-indicator analysis for comprehensive performance assessment

Taking all of these considerations and pre-work into account, the Value Creation Model was then used to design a one-page “KPI Dashboard”, allocating the most strategic indicators to the respective factors of the model (see Figure 15), with the aim to have a full standardized overview and assessment of the actual performance of each ISI at hand, to be used as a management and communication tool inside the SENAI organization.

Figure 15
KPI Dashboard with National Standard Indicators for Strategic Performance Assessment (Example).

These KPI Dashboards then served to analyze the performance of each institute regularly, i.e. every three months the respective data was reported to SENAI’s national department by the Innovation Institutes, and a mixed team of analysts of SENAI and Fraunhofer IPK assessed these data remotely at the headquarters. On a first analysis level only four basic indicators, highlighted in blue on the right side of the KPI Dashboard (see Figure 15), were being assessed to generate a first overview of the financial and managerial “health” of the respective institute. These four basic indicators, basically measuring the achieved business results of each institute (Business Model dimension “Business Success”), allow a first high-level assessment based on the following interpretation context (see Table 1):

Table 1
Four basic KPIs for first-level performance analysis.

Business Model Factor (Business Success)	Basic KPI	Interpretation of KPI
Image & Reputation	Total Revenue (R$)	The absolute total revenue of each SENAI Innovation Institute (ISI) indicates the relevance of the institute’s work on the market in the overall. A minimum size of the institute (measured by its total revenue), to ensure visibility and recognition by the market, needs to be achieved by each ISI.
Financial Sustainability	Costs Covered by Revenue (%)	To measure the general financial health of the institutes, the share of the operational costs covered by revenue shows the degree of the respective ISI’s financial sustainability. An average of 100% has been set as the national target to ensure the financial autonomy of each Innovation Institute.
Growth of National Innovation System	Share of R&D&I Revenue (%)	As ISIs are allowed to also offer basic technological services (metrology, consultancy etc.), the share of revenue made by research, development and innovation projects measures to which degree the ISI is behaving as a “real” innovation institute, and thus, contributing to the national advancement in applied R&D. For full-grown institutes a minimum share of 70% has been defined as a national requirement.
Competitiveness of Industry	Share of Industry Revenue (%)	The share of revenue, coming directly from industrial clients, indicates the relevance of the ISI’s service offerings for the industry, and thus, the perceived value of ISI’s contributions to the industry’s competitiveness. Individual targets are agreed in the Management Audit, respecting a minimum threshold of 30%.

Depending on the result of this First-Level Performance Analysis, the additional KPIs of the dashboard are used to detect possible causes for a certain performance gap in the business results. If, for example, the KPI “Share of Industry Revenue (%)” does not show the desired level of business success (the target value being derived from the strategic objectives of the network and the individual institute), the reason for this under-performance could be a) unattractive R&D offerings and/or unclear business benefits (Customer Value), or b) missing systematic procedures in “Marketing & Sales” for pro-active acquisition of new industry clients (Business Processes), or c) a lack of communication competence and experience in working with industrial companies (Human Capital), or d) a mix of these and other possible reasons. The indicators allocated in the respective pillars of the Value Creation Model may give first hints on the real cause of a certain performance gap, but a qualitative investigation always remains necessary to validate and further analyze a certain management challenge. However, in the shown approach, this investigation can be executed as efficient and focused as possible by starting the performance analysis always from the top level of the business results and, only in case of detected performance gaps, going to deeper levels of the Value Creation Model in a directed way, i.e. in search of evidence and causes for challenges on the top level.

Therefore, and additionally to the quantitative KPI performance analysis based on regular reports of the institutes’ current KPI values and targets, SENAI’s national department requested a yearly update of their qualitative strategic planning regarding the Business Model levels “Markets” and “Products & Services” (see chapter 4.1), i.e. adjustments concerning the focused market segments and the main service areas of the Innovation Institute based on learnings from the market, changes in the business environment and/or internal changes affecting the institute’s strategy. Besides that, formal evidence for the minimum requirements set up by the Maturity Check (see chapter 4.2) was requested as a preparation for the Management Audit. With this input at hand before the actual audit, the moderated discussions during the Management Audit then focused on the verification of the current status and actual performance of the respective Innovation Institute in the light of the previously revised strategic objectives. As a result, the right strategic actions to close individual performance gaps and tackle prioritized management challenges of the institute could be derived and defined.

In the Strategic Action Map those actions are allocated inside the Value Creation Model and interdependencies of the driving factors and desired results (targets) are displayed by directed connections between the Business Model factors indicating specific cause-and-effect chains of the institute’s individual strategy (see Figure 16). Following the basic logic of the Value Creation Model and of the performance analysis described above, the actions are allocated on the left side and aim at closing gaps in Resources/ Intellectual Capital or improving certain performance aspects of Business Processes, to produce certain results and improvements on the right side of the model, i.e. the desired Business Success, including defined revenue targets and target values of other basic KPIs.

Figure 16
Strategic Action Map with Mid-Term Actions and Performance Targets (Example).

With the Management Audit at its core, the final Evaluation System for the “Stable Operations Phase” of the SENAI Innovation Institutes reflects a lean and pragmatic approach comprising all important information and content to generate a comprehensive and in-depth view of the current situation, i.e. the institutes “managerial health” and the necessary strategic steps for continuous growth. In a one-day workshop procedure, it is possible to generate a full and comprehensive overview of the institute’s actual status, its challenges and to create the right action plan for the strategic route each Innovation Institute strives to pursue in a systematic way. This on-site audit is complemented by the Maturity Check to guide through the stage-gates of the Ramp-up Phase, ensuring certain minimum requirements of the national network of Innovation Institutes, and by the regular reporting of the standard KPIs to continuously monitor the performance of all institutes. With these elements established and working, this Management Evaluation System now allows a constant tracking of the evolution of each institute and of the network as a whole, providing objective data and analysis results on performance and management challenges to react quickly and well-directed when significant deviations from the set targets in one of the Business Model dimensions are being detected.

This Management Evaluation System, enhanced by an evaluation of the technological maturity of the Innovation Institutes (not subject of this article), serves as the main building block for the network governance and the respective management system of SENAI to be used for the further strategic development of this newly established national network of applied R&D institutes in Brazil.

5. Results and lessons learned

After the first 5 years of ramping-up the operations, the 25 SENAI Innovation Institutes are counting on the workforce of more than 650 researchers and specialists (approx. 300 of them having a master or doctor degree), supported by a large technical and administrative team at the institutes, as well as at the regional departments and the national department, having already acquired and executed R&D projects for the industry with a total economic value of more than R$ 750 million (approx. 200 million USD). With these impressive growth indicators and a well-targeted strategic development of the ISI network, SENAI is in the process of achieving its overall objectives, helping to transform the industry in Brazil towards a higher level of competitiveness and productivity through innovation and the implementation of new technologies.

After guiding the institutes through the initial planning and ramp-up phase, the main element of the governance and management system for the full-grown national network of the 25 SENAI Innovation Institutes in the “Stable Operations Phase” is the integrated Evaluation System, including the Management Audits described above. The on-site Management Audit is being performed every two years in the regular approach and/or in the case of significant deviations or managerial challenges identified by the regular remote performance analysis (quarterly, yearly). Following the methodological requirement of implementing a lean and pragmatic evaluation system, this audit procedure is possible to be executed in one full day, for more mature institutes and after an initial learning curve it may be reduced to a half-day workshop program. In a last evolutionary step, the main results of this management evaluation are now condensed into one A3-page canvas-like overview which forms the basis of the “management pact” between the Innovation Institute, the respective regional department’s directorate and SENAI’s national department.

One important learning is that the personal discussions between the institute’s management staff and the regional and national department of SENAI at the on-site Management Audit, moderated by neutral management experts and coaches, are crucial for a valid and agreed assessment of performance gaps, for an in-depth investigation of the real causes for these challenges and for the derivation of adequate and feasible actions to tackle these individual gaps and challenges. It is worthwhile noting that the standardized management framework, model and tools including the standardized KPI system work well to streamline the evaluation process, generating verifiable and agreed results regarding the strategic development of each institute, but these data and numbers will never automatically allow a full understanding of the particularities at the institute’s site itself, and can never substitute a deep discussion process between the responsible management staff and experienced experts to elaborate the right measures and actions together, creating a common understanding of all involved parties for the reasons and importance of certain actions, which may require investments from the mother organization. Thus, one important purpose of the described models, tools and procedures is to structure and systematize this internal discussion process and to serve as communication instruments inside the organization to guarantee this common understanding.

Besides many external obstacles in the dynamic Brazilian market and political system, one important challenge was and is the adaptation of the mother organization towards the innovation business. Taking into account its 70-year history as a large national organization for technical education, it is somewhat natural that an organization of that size with almost no prior experience in the field of R&D and innovation has to overcome certain barriers of behavioral change inside the various departments and functions that all need to support this highly dynamic and challenging business of applied research for the industry, including HR, finance, legal, purchase, communication and many other supporting departments which still need to be reoriented and empowered to include the requirements of the ISI network in their daily work. This challenge will need a continuous effort in the next phase of the lifecycle of this newly created national network of Innovation Institutes for the Brazilian industry.

While this article has clearly focused on the managerial aspects of planning, implementing and evaluating a national network of applied research institutes in an emerging innovation system, the technological part of this endeavor can, of course, not be neglected. Therefore, the fully integrated Evaluation System also includes a procedure for continuously tracking the technological maturity of each institute and a respective Technology Audit (Hecklau et al., 2019). As an outlook, it may be stated that SENAI and Fraunhofer IPK are planning to enhance this Evaluation System even further in the future, integrating measurements and analysis of impact in the National Innovation System, i.e. for investigating advancements in research and technology as well as in industrial performance on a regional and national level in Brazil.

• How to cite this article: Kohl, H., Will, M., Prim, M. F., Pavim, A. X. (2020). Building up a national network of applied R&D institutes in an emerging innovation system. Production, 30, e20190151. https://doi.org/10.1590/0103-6513.20190151

• Erratum

  In the article Building up a national network of applied R&D institutes in an emerging innovation system, DOI number: http://dx.doi.org/10.1590/0103-6513.20190151, published in journal Production, 30: e20190151, page 1:
  Where it reads:
  Markus Will^a*, Holger Kohl^a,b, Marcelo Fabricio Prim^c, Alberto Xavier Pavim^c
  It should read:
  Holger Kohl^a,b, Markus Will^a*, Marcelo Fabricio Prim^c, Alberto Xavier Pavim^c
  Where it reads:
  How to cite this article: Will, M., Kohl, H., Prim, M. F., Pavim, A. X. (2020). Building up a national network of applied R&D institutes in an emerging innovation system. Production, 30, e20190151. https://doi.org/10.1590/0103-6513.20190151
  It should read:
  How to cite this article: Kohl, H., Will, M., Prim, M. F., Pavim, A. X. (2020). Building up a national network of applied R&D institutes in an emerging innovation system. Production, 30, e20190151. https://doi.org/10.1590/0103-6513.20190151

References

• Alwert, K., Bornemann, M., & Will, M. (2008). Wissensbilanz – Made in Germany. Leitfaden 2.0 Berlin: German Federal Ministry for Economics and Technology Germany.
• Alwert, K., & Will, M. (2014). Leitfaden Maßnahmen managen. Zusatzmodul zum Leitfaden 2.0 zur Erstellung einer Wissensbilanz Berlin: Fraunhofer IPK.
• Barbieri, J. C., & Álvares, A. C. T. (2016). Sixth generation innovation model: Description of a success model. Revista de Administração e Inovação, 13(2), 116-127.
• Barney, J. (1991). Firm resources and sustained competitive advantage. Journal of Management 17(1), 99-120. http://dx.doi.org/10.1177/014920639101700108
  » http://dx.doi.org/10.1177/014920639101700108
• Battista Dagnino, G., Levanti, G., Minà, A., & Massimo Picone, P. (2015). Interorganizational network and innovation: A bibliometric study and proposed research agenda. Journal of Business and Industrial Marketing, 30(3-4), 354-377. http://dx.doi.org/10.1108/JBIM-02-2013-0032
  » http://dx.doi.org/10.1108/JBIM-02-2013-0032
• Becker, T., Dammer, I., Howaldt, J., & Loose, A. (2011). Netzwerkmanagement: Mit Kooperation zum Unternehmenserfolg Berlin, Heidelberg: Springer-Verlag.
• Chesbrough, H. W. (2003). Open Innovation: the new imperative for creating and profiting from technology. Harvard Business School; Boston, Mass Maidenhead McGraw-Hill.
• Corsten, H., Gössinger, R., & Schneider, H. (2006). Grundlagen des Innovationsmanagements München: Vahlen.
• Coughlan, P., & Coghlan, D. (2002). Action research for operations management. International Journal of Operations & Production Management, 22(2), 220-240. http://dx.doi.org/10.1108/01443570210417515
  » http://dx.doi.org/10.1108/01443570210417515
• Drucker, P. F. (1985). Innovation and entrepreneurship: Practice and principles London: Heinemann.
• European Commission (2008). InCaS: Intellectual Capital Statement – Made in Europe. European ICS Guideline Retrieved in 2018, December 4, from http://www.incas-europe.org
  » http://www.incas-europe.org
• European Foundation for Quality Management (2010). EFQM Excellence Model: EFQM Model 2010 Brussels: EFQM.
• European Research Advisory Board (2005). Research and Technology Organisations (RTOs) and ERA. Final Report. Retrieved in 2018 April 24, from http://ec.europa.eu/re-search/eurab/pdf/eurab_07_07_may_2007_en.pdf
  » http://ec.europa.eu/re-search/eurab/pdf/eurab_07_07_may_2007_en.pdf
• Freeman, C. (1987). Technology policy and economic performance: Lessons from Japan London: Pinter.
• Hauschildt, J., Salomo, S., Schultz, C., & Kock, A. (2016). Innovationsmanagement (6th ed.). München: Franz Vahlen. http://dx.doi.org/10.15358/9783800647293
  » http://dx.doi.org/10.15358/9783800647293
• Hecklau, F., Kidschun, F., Will, M., Kohl, H., Prim, M. F., Pavim, A. X., & Oliveira, J. E. (2019, September 19-20). Application example: assessment of the technological maturity of brazilian innovation institutes. In P. Liargovas & A. Kakouris. Proceedings of the 14th European Conference on Innovation and Entrepreneurship (ECIE) Reading, UK: Academic Conferences and Publishing International Limited.
• Henderson, B. D. (1970). The product portfolio. In C. W. Stern & G. Stalk (Eds.), Perspectives on strategy New York: John Wiley and Sons.
• Kaplan, R. S., & Norton, D. P. (1996). Balanced scorecard: translating strategy into action Boston: Harvard Business School Publishing.
• Kaplan, R. S., & Norton, D. P. (2004). Strategy maps: Converting intangible assets into tangible outcomes Boston: Harvard Business School Press.
• Kohl, H., Orth, R., Riebartsch, O., & Hecklau, F. (2016). Integrated Evaluation System for the Strategic Management of Innovation Initiatives in Manufacturing Industries. Proceedings of the 13th Global Conference on Sustainable Manufacturing Amsterdam: Elsevier B.V. http://dx.doi.org/10.1016/j.procir.2016.01.057
  » http://dx.doi.org/10.1016/j.procir.2016.01.057
• Kozioł-Nadolna, K., & Świadek, A. (2010). Innovation process models with emphasis on open innovation model. Folia Oeconomica Stetinensia, 9(1), 167-178. http://dx.doi.org/10.2478/v10031-010-0007-5
  » http://dx.doi.org/10.2478/v10031-010-0007-5
• Kuhlmann, S., & Holland, D. (1995). Erfolgsfaktoren der wirtschaftsnahen Forschung. Fraunhofer ISI Heidelberg: Physica-Verlag. http://dx.doi.org/10.1007/978-3-642-52412-7
  » http://dx.doi.org/10.1007/978-3-642-52412-7
• Leydesdorff, L., & Etzkowitz, H. (1995). The Triple Helix of University-Industry-Government Relations: A Laboratory for Knowledge-Based Economic Development. European Association for the Study of Science and Technology Review, 14(1), 14-19.
• Mobilização Empresarial pela Inovação (2018a). Desempenho do Brasil no Índice Global de Inovação 2011-2018 Brasília: CNI.
• Mobilização Empresarial pela Inovação (2018b). A MEI e o Desafio da Inovação no Brasil – Um Balanço de 10 Anos de Avanço Brasília: CNI.
• Mertler, C. A. (2017). Action research: improving schools and empowering educators Thousand Oaks, California: Sage Publications. http://dx.doi.org/10.4135/9781483396484
  » http://dx.doi.org/10.4135/9781483396484
• Mowery, D. C., & Rosenberg, N. (1993). The U.S. National Innovation System. In R. Nelson (Ed.), National Innovation Systems: A Comparative Analysis New York: Oxford University Press.
• Müller-Prothmann, T., & Dörr, N. (2014). Innovationsmanagement: Strategien, Methoden und Werkzeuge für systematische Innovationsprozesse 3rd ed. München: Hanser. http://dx.doi.org/10.3139/9783446439337
  » http://dx.doi.org/10.3139/9783446439337
• Organisation for Economic Co-operation and Development (2002). Frascati Manual: Proposed standard practice for surveys on rese-arch and development. Paris: OECD Publishing.
• Organisation for Economic Co-operation and Development (2011). Actor Brief on Public Research Organizations Paris: OECD Publishing. Retrieved in 2019, March 23, fromhttp://www.oecd.org/innovation/policyplatform/48136051.pdf
  » http://www.oecd.org/innovation/policyplatform/48136051.pdf
• Ozman, M. (2009). Inter-firm networks and innovation: A survey of literature. Economics of Innovation and New Technology, 18(1), 39-67. http://dx.doi.org/10.1080/10438590701660095
  » http://dx.doi.org/10.1080/10438590701660095
• Porter, M. E. (1996). What is Strategy? Harvard Business Reviev, 74(6), 62-78.
• Ranga, M., & Etzkowitz, H. (2013). Triple Helix systems: An analytical framework for innovation policy and practice in the Knowledge Society. Industry and Higher Education, 27(4), 237-262. http://dx.doi.org/10.5367/ihe.2013.0165
  » http://dx.doi.org/10.5367/ihe.2013.0165
• Reynolds, E. B., Schneider, B. R., & Zylberberg, E. (2019). Innovation in Brazil – Advancing Development in the 21st Century New York: Routledge. http://dx.doi.org/10.4324/9780429053092
  » http://dx.doi.org/10.4324/9780429053092
• Ringwelski, J. (2017). Emergenz und Persistenz staatlich geförderter mittelständischer Innovationsnetzwerke und ihrer Promotoren (Dissertation). Technische Universität Berlin: Berlin.
• Roll-Hansen, N. (2009). Why the distinction between basic (theoretical) and ap-plied (practical) research is important in the politics of science London: The London School of Economics and Political Science, Centre for the Philosophy of Natural and Social Science Contingency and Dissent in Science Technical Report.
• Schumpeter, J. (1912). Theorie der wirtschaftlichen Entwicklung Leipzig: Duncker & Humblot.
• Sydow, J. (2010). Management von Netzwerkorganisationen - Beiträge aus der “Managementforschung” (5th ed.). Wiesbaden: Gabler.
• Taferner, B. (2017). A next generation of innovation models? An integration of the innovation process model big picture towards the different generations of models. Review of Innovation and Competitiveness, 3(3), 47-60. http://dx.doi.org/10.32728/ric.2017.33/4
  » http://dx.doi.org/10.32728/ric.2017.33/4
• Tripp, D. (2005). Action research: a methodological introduction. Educação e Pesquisa, 31(3), 443-466. http://dx.doi.org/10.1590/S1517-97022005000300009
  » http://dx.doi.org/10.1590/S1517-97022005000300009
• Will, M. (2012). Strategische Unternehmensentwicklung auf Basis immaterieller Werte in KMU - Eine Methode zur Integration der ressourcen- und marktbasierten Perspektive im Strategieprozess Stuttgart: Fraunhofer Verlag.
• Will, M. (2020). Integrated Strategy Development Based on Intangibles. In P. Ordónez de Pablos & L. Edvinsson (Eds.), Intellectual capital in the digital economy New York: Routledge.
• Will, M., & Wuscher, S. (2014). Leitfaden Strategische Ziele entwickeln. Zusatzmodul zum Leitfaden 2.0 zur Erstellung einer Wissensbilanz Berlin: Fraunhofer IPK.
• Zink, K. J. (2004). TQM als integratives Managementkonzept. Das EFQM Excellence Modell und seine Umsetzung 2nd ed. München: Hanser. http://dx.doi.org/10.3139/9783446228290
  » http://dx.doi.org/10.3139/9783446228290
• Zylberberg, E. (2017). Beyond RTO Benchmarking: Towards a typology of innovation intermediaries (MIT-IPC Working Paper, 17-002). Cambridge: MIT-IPC Working Paper Series.

Publication Dates

• Publication in this collection
  05 Mar 2020
• Date of issue
  2020

History

• Received
  02 Dec 2019
• Accepted
  02 Feb 2020