Impact evaluation for University-Business Cooperation and Technology Transfer in higher education systems: cluster analysis

Abstract

Higher education systems evolved in recent decades. Universities must not only provide society with capable professionals but also act in the market for technologies, knowledge, and ideas to promote technological development. This paper discusses the motivational performance evaluation system for technology transfer process, specifically the patterns’ evaluation of academic units considering micro-cultures and idiosyncrasies’ analysis, in the academic context of autonomy. Based on action research, the existing performance evaluation system was assessed, and multivariate cluster analysis was proposed and tested as a method to enable micro cultures’ identification and evaluation. The analysis proposed enabled a tool for reflexive discussion regarding the effectiveness of the institutional innovation system in academic units and Engineering Education, and its implications for social and technological development of industry and society enabled action proposals for improvement in the university’s technology transfer management process.

Keywords
Technology management; Technological innovation; Patents; Performance measurement system; University-industry interaction

1. Introduction

Higher education systems have evolved in the last decades. Universities not only provide society with qualified professionals but must also play a role in the market for technology, knowledge, and ideas to promote technological development. Technology transfer (TT) from universities comprises the main mechanism for regional development. Practical action such as regulations and legislation is observed in several countries, in so-called Bayh-Dole-like acts (Argyres & Liebeskind, 1998; Etzkowitz & Leydesdorff, 2000; Grimaldi et al., 2011; Nārāyaṇan & O’Connor, 2010; Niosi, 1999; Shane, 2004; Wright et al., 2004).

The TT public policy perspective has been largely discussed (Bozeman, 2000). Contemporary public management view considers all stakeholders, researchers, managers, and employees as active agents, with a role in social and technological development (Wieczorek & Hekkert, 2012). Effective technological development will not occur unless research institutions and its administrative and academic units have been structured for effective TT.

The TT process and its complexity must be understood and monitored aiming adaptation to technology related dynamic environments. Although largely used (Bubela & Caulfield, 2010), performance assessment (PA) based purely on the amount of generated patents, spin-offs, and TT projects is limited (Agrawal & Henderson, 2002). They are macro and aggregated indicators, and, although they enable comparison between institutions, are not effective to reveal researchers’ interaction with industry and its impact on research output (Perkmann & Walsh, 2009). This work discusses data analysis methods for monitoring the technology transfer (TT) process, rescuing decision theories and performance management systems (PMS), related the discussion to evaluate the applicability of multivariate methods (specifically cluster analysis).

2. Literature

This work is based on PMS and strategic decision, contextualized in the TT and Institutional Innovation systems. These topics are discussed shortly as follows.

2.1. Performance management systems, institutions, and innovation system

Institutions influence organizations’ behavior, shape, and structure. They define organizational practices as acceptable, reinforce these definitions, usually restraining organizational resources. PMS usually comprise a definition of a socially elaborated effectiveness construct for its development, considering also participant satisfaction or strategic constituency (Breschi & Malerba, 1997; Coriat & Weinstein, 2002; Hiatt et al., 2009; Meyer & Scott, 1983; Meyer & Rowan, 1977; Quinn & Rohrbaugh, 1983; Scott, 1995).

PMS are used to enable companies and organizations to learn, evolve and grow. However, new organizational forms, which involve a network and virtual connections, co-development, open innovation (OI), corporate or academic spin-offs, joint-venture, present less routinized or foreseeable processes and systems, as R&D processes. Traditional managerial models and PMS are not well-suited for this context (Chiesa & Frattini, 2009; Kerssens-van Drongelen & Cook, 1997; Lehtinen & Ahola, 2010; Phusavat et al., 2009; Pillai et al., 2002).

The literature argues that especially in a context where the staff presents freedom to translate organizational inputs into output, organizational performance evaluation must not only count on the basis of programs’ outcomes but also on the adequacy level of management structures (Selden & Sowa, 2004). Academic discussions since 2000 improve the comprehension of the PMS development process as a cognitive process (Taticchi et al., 2010), comprising not just controlling process, but also learning, decision making, and motivation processes (Kerssens-van Drongelen & Cook, 1997).

These discussions focus on the effectiveness of the existing motivational mechanisms, considering organizational and decision theories. The complex processes related PMS comprise strategic decisions, defined as a wicked problem. Decision process design view enables the flexibility needed for an evolving system and considers the organizational bounded rationality (behavioral theory for organizations and decisions), human behavior, memory, experience, group related issues, incentives, authority, organizational influence, and identity. In this context, the decision is based on collection and consolidation of individuals’ perception. The genuine decision for wicked problems comprises new knowledge discovery. Several information sources, both qualitative and quantitative data, are required. No optimal solution is possible, and reality representation models and visualization tools enable discussions and the search for a satisfactory solution (Ackermann & Eden, 2010; Ackoff, 1979, 1981; Byström & Järvelin, 1995; Checkland, 2000; Keller & Tergan, 2005; Lurie & Mason, 2007; Mintzberg & Lampel, 1999; Pidd, 2008, 2010; Power, 2008; Simon, 1979, 1997; Tergan et al., 2006).

In the practical domain, some considerations regard evidence collection and use for evidence-based PM and managerial learning (Rousseau, 2006). In complex processes and phenomena, univariate controls become useless. The use of available data collection and multivariate methods for data analysis and knowledge discovery is discussed as an interesting tool for organizational intelligence for strategical definitions and business development (Adams et al., 2000; Meisel & Mattfeld, 2010; O’Keefe & Preece, 1996).

2.2. Motivation and performance in Technology transfer

Motivation drivers influence the occurrence of different TT mechanisms in the University. Although the beneficial feedback loops between technology commercialization and scientific research performance are discussed, this is not a consensus. The literature warns about entrepreneurial university promotion intended undifferentiated public policy definitions, because putting a high value on IP protection, technology commercialization, and goals focused on income maximizing, can demote learning promotion objectives. The IP licensing cases are usually modest, and there is evidence that collaborative R&D projects with industries are more consistent with both scientific and technology development and commercialization. The human behavior comprises different motivational issues, such as reputation, monetary profit, the possibility of acquisition of additional research funds, learning from industry, access to in-kind resources (Agrawal & Henderson, 2002; D’Este & Perkmann, 2011).

There are several pieces of evidence regarding differences between knowledge domains, university departments, and industries. It is known that patent licensing is more frequent in some areas, such as pharmaceutical, computer, and electronics industry (Guellec & Ménière, 2014). Domains related to biotechnology and pharmaceutical industries consider patents as a potential income enhancer because they are monetarily more valuable (D’Este & Perkmann, 2011). Also, different firms acquire knowledge by different channels, and there are differences in faculty behaviors regarding patenting (Agrawal & Henderson, 2002). Hence, there is sufficient evidence to consider microcultures or idiosyncrasies in the efficiency definition by PMS.

3. Methodological considerations

This paper presents a qualitative and quantitative approach based on action research to discuss PMS for Technology Transfer process in universities. Technology Transfer Office (TTO) available databases were grouped, treated and mined to evaluate the relevant characteristics of the university’s academic units. The proposed method was discussed for its usefulness and relevance of micro-cultures assessment, to promote improvement opportunities for the university TT process management.

4. The context

Brazilian universities, as other universities from other countries, have multiple and imprecise objectives. Although research is one of the objectives, no staff members are hired exclusively as a researcher. They are hired as tenured professors (Leitão, 1987) to conduct teaching activities in a specific academic unit, which is a formal and permanent subdivision of the studied University. The professionals’ autonomous engagement in multiple academic units, participating in multidisciplinary centers or graduate programs, is essentially a concession from their original academic unit. Furthermore, they are free to decide (i) if they engage in research, (ii) in which research lines to invest, (iii) if this research is conducted with or without interaction with the productive sector, (iv) if the resulting intellectual property will be protected.

In the last decades, universities’ management and performance management discussions inducted more consistent strategic planning tools (Segenreich, 2005). Although it has still to improve, it shows a more proactive view of modern public management (Azevedo, 1992; Fortis, 2010; Hood, 1995; Matus, 2006). But, actually, undergraduate courses are evaluated, not universities (Barreyro & Rothen, 2008). Technology generation is complementary data in this evaluation, and commercialization is not specifically evaluated. As in other countries, TT metrics for universities and research institutes comprise filed patents amount.

5. Case for Analysis

The study was applied to a traditional Brazilian public University with relevant research history. The university presented 2081 hired professors, 322 registered patents and an average of 509 contracts per year in the last years. The Institutional Development Plan (PDI) led each academic unit and managerial department to develop their own indicators. Usually, managerial departments (including TTO) report their own activities and results, although some global results are also included. The indicators for whole system output, when existent, are just informative. These global indicators do not deliver actions for improvement because they are established for each department, not for a process, and usually, the original department does not have the power to impose actions for other organisms and academic units.

In Brazil, usually discussions regarding university improvements and evaluation mention university autonomy (Barreyro & Rothen, 2008; Leitão, 1987; Zainko, 2008). It is a consensus that professors value autonomy and dislike bureaucratic authority (Leitão, 1987). The TT is not a compulsory activity, and there are researchers that even consider it inappropriate.

We included patent generation in the PMS as a type of output-based quality measure for research, enabling comparison between institutions by public agencies. But in the broader context of TT and commercialization, it is just an intermediate, not revealing how researchers interact with industry and its impact on research output (Perkmann & Walsh, 2009). The real performance and outcome of this process would be observable only in the long term, with regional or national technological development, which is difficult to evaluate.

The financial perspective is very hard to measure in the complex and variable system comprising the University. Because of process complexity and variability, it is not possible to define a plausible efficiency measure. Although the described scenario indicates that it is yet in a state of transition and the University has much to learn and innovate in its processes and systems, this innovativeness and learning is not specifically measured. The evaluation of federal university’s courses inspires learning by self-evaluation, but the organizational learning processes are not effectively measured.

Regarding the PMS system classification, the patent and publication amount focus aims goal alignment and communication, which characterizes soft objectives (Chiesa & Frattini, 2009). The opportunity to earn royalties through technology licensing may be analyzed as a kind of reward, what would comprise a hard objective. However, licensing is not so common and not granted for all registered IP. Hence, a direct monetary reward is practically absent, but this also comprises one of the criteria for career progression, that could be characterized as a hard objective of PMS.

Registered IP amount represents object control based on output because it is quantitatively measurable. Regarding control type, this measure has the objective of feedback control, aiming to explore the real performance measures and objectives comparison (Kerssens-van Drongelen & Bilderbeek, 1999), using bibliometrics and IP register data. However, as previously mentioned, considering the context of TT only as an intermediate for the regional and national technological development process, they are not the real PA.

5.1. Definition of PMS goals and configuration

The main premises and observations and considered implementation issues are presented in the Table 1. The diagnosis showed the loss of systemic view, a classic problem in large and complex organizations (Burbidge, 1995; Voss, 2005). Although the PDI was elaborated deploying organizational objectives, the absence of the systematic, process or value chain view does not enable a full use of the potential of established PMS, specifically, to enable feedback for process improvement.

Table 1
Premise, observations, and defined PMS implementation issues.

Premise and/or observation	implementation issue
Incipiency and low managerial maturity	Need for flexibility in the system and learning objective – knowledge discovery
Complexity and uncertainty of the innovation system, its scattered and intangible nature -,
Absence of authority to charge people, as they consider themselves autonomous and the TT engagement an elective activity	PMS with a motivational objective
The absence of effectively institutionalized control system regarding TT engagement
Difference between departments and academic units because their knowledge domain and related industry issues	Measurement objective- identification of microcultures or idiosyncrasies, and their impacts on the process
Observation: focus on subgroups comprised by academic units of the institution, because decision making usually occurs in this level	Monitoring unit- subgroups comprised by academic units of the institution

The motivational objective is adequate for the analyzed context, as PMS can provide diagnosis and further discussions for improvement. The monitoring unit comprises the subgroups comprised by academic units, because decision making usually occurs at this level. The measurement objective comprises the identification of microcultures or idiosyncrasies and their impacts on the process. The proposition was applied in the case’s originated data, in the following.

5.2. Data treatment and mining on TTO available data

The data covered 1032 knowledge transfer university-company interaction projects (contracts) and 121 patents’ registration processed by the TTO from January 2009 to October 2012. The database considered only cooperative projects with some kind of technological transfer. The data were treated to obtain evaluation indexes considering the performance constructs presented in the Table 2.

Table 2
Performance constructs.

Construct	Dimension
Project effectiveness for the Patent generation	the inclination of the academic unit’s researchers to transfer knowledge
	effectiveness to generate intellectual property from conducted projects’
	generation of intellectual property (IP), alone or with the productive sector
Commercialization potential	generation of intellectual property (IP), with the productive sector
	generation of intellectual property (IP), alone or with other entities and academic units
Project engagement	the inclination to use specific knowledge transfer mechanisms
Partnership profile	the inclination to interact with productive and public sectors

The cluster analysis was considered adequate, considering the micro-cultures and idiosyncrasies identification premise. The variation of academic units in size and number of researchers were accounted by relativizing the variables to enable comparison. For multivariate cluster analysis application, care was taken to assure a similar amount of variables for every conjoint, aiming to avoid dominance of one theme in the clusters establishment. The set of variables (described in ^{Appendix A}) presents an apparent redundancy because the complementarities needed for construct assessment.

The variables also estimate the effectiveness in patents generated related to a number of project categories, and the involvement of productive and public sectors. A refinement comprised by commercialization potential of patents is also presented, as related to joint ownership with the productive sector indicative of the likelihood of alignment with market needs, or multidisciplinarity represented by patents registered with the involvement of at least one other academic unit.

Cluster analysis was conducted using Ward method and squared Euclidian distance as a similarity measure. Considering the difference regarding a number of projects and registered patents of the academic units, all variables were defined as percentages, and also standardized by Z-score, to enable comparability of the variables. Figure 1 presents the dendrogram with the clustering pattern of academic units.

Figure 1
Dendrogram with clusters of academic units, using Ward Linkage.

Table 3 describes the obtained clusters. The description is based on the significant indexes defined by Kruskal-Wallis independent test in the appendix section (with descriptive graphs).

Table 3
Obtained clusters.

Cluster	Description
Knowledge transfer by teaching activities cluster (C1)	The lacks patent registration and interaction culture with public or productive sectors.
Multidisciplinary cluster (C2)	Some generation of patents, with emphasis to multidisciplinary patents.
high generation of patents (C3)	Stands out in R&D projects and patent generation
Extension cluster (C4)	High knowledge transfer, but with an inexpressive number of registered patents; Knowledge transfer projects are focused on teaching and extension (significantly lower occurrence of R&D projects) and present insignificant interaction with the public and productive sectors
influenced by the public sector (C5)	emphasis on R&D projects, and apparently has significant influence of the public sector to establish these projects

5.3. Knowledge/ hypothesis/ question generation from the analysis

Comparison of clusters description, the grouping pattern discussion, and other complementary data or information enabled the knowledge discovery. They were useful for managerial considerations, comprising some hypothesis for further investigation regarding current managerial practices and public policies. Table 4 presents the main observations described in following.

Table 4
Main observations and knowledge discovered.

Source/ Observation	Complementary data/information	Generated knowledge
Ratio P.ProdPublProj:P.ProdProj	-	Project engagement pattern Diagnosis
patent/R&D project ratio	Premise: the patent/R&D project ratio must be smaller than one (1) to be logic	Indication of investment in registering purely academic patents
JOPatent_ProjProdS; P.MDiscPatent		Significant amount of patents with low commercialization potential
C2 profile x Expected profile: generation of patents, with emphasis to multidisciplinary patents	Patent registration data - in Brazil	Apparently a benchmark group in the institution
	Premise of Relevance of Multidisciplinarity for TT efficiency
Composition of C1 & Comparison of C1 and C2 profiles: expected to be included in the cluster 2: Food Science and Technology and Veterinary Medicine - low patent generation	Patent registration data - new technologies are developed in this knowledge area in a global context	These productive sectors are not demanding new technological knowledge? or the academic units are less inclined to interact with the productive sector?
	TTOs experience and perception regarding local reality	Insufficient multidisciplinarity – especially with biotechnology department
C3 profile analysis & Comparison with expected profile: Lifesciences, engineering, physics, chemistry, and pharmacy – high patent generation	National Patent registration data - Pharmaceuticals, Organic Fine Chemistry and Basic Materials Chemistry are the most significant technology areas	Patent registering stimuli is making results
C3 profile analysis x expected profile: Lower amount of conjoint patent ownership with productive sector of the Engineering; R&D projects with community are slightly lower than C5	Bourelos et al. (2012) - research in engineering more directly applicable to firms than is the case in other disciplines; less pharmaceutical and biotech companies in Brazil than engineering companies	Low applicability of conducted R&D projects? low interest of local industry for technological innovation? or low conscience of local industries about the university as a source of knowledge or technology?
C3 profile, Isolated P.ProdPublProj: P.ProdProj ratio graph (Figure 2): interaction with the productive sector is low if the public sector is absent.		Public policy is acting to stimulate TT process. The long-term policies and strategies must deal about the evolution of a relation more autonomous, that occur even without public incentive
Composition of C3 & Comparison of C3 and C2 profiles: It was expected a more multidisciplinary behavior the following academic units		The applied nature of the Engineering and Pharmacy academic units are inducing a sense of self-efficiency?
C4 profile and composition x expected profile: Dentistry - inexpressive number of registered patents; extension activities to provide services to the community	applied nature of knowledge produced by Dentistry course would be expected to generate research fields useful for development of dental products that could be patented	Lower culture for technological transference, with emphasis on extension activities to deliver established techniques for the community. Claims for managerial attention.
C5 composition and profile x Expected patent generation amount: low patent generation of informatics and Hydraulic Research Institute	Patent registration data – higher patent registration Digital Communication, Computer technology, Semiconductors, IT Methods for management); Civil Engineering	Low tendency to protect IP, even with high innovativeness? Or Low innovativeness?
C5 composition and profile x expected profile Isolated P.ProdPublProj: P.ProdProj ratio graph (Figure 2): interaction with the productive sector is low if the public sector is absent.	Project short description (public departments, such as Sanitation and Energy, and public institutions are main partners)	Public policy is acting to stimulate TT process. The long-term policies and strategies must deal about the evolution of a relation more autonomous, that occur even without public incentive

• C1 - Knowledge transfer by teaching activities cluster; C2 - multidisciplinary cluster; C3 -high generation of patents cluster; C4 - Extension cluster; C5 - cluster influenced by public sector.

5.3.1. Unidimensional and transformed dimensions analyses

Although unidimensional and multidimensional evaluations proved to be useful, because of the complexity dealing potential, the useful indexes were the already transformed ones, derived from at least two original indexes. They are described in following as “Partnership profile & Project engagement pattern” and “TT System Effectiveness, Quality and Impact evaluation”.

Partnership profile construct, defined as an inclination to interact with productive and public sectors, was evaluated considering the P.ProdPublProj (projects with involvement of both public and productive sector) and P.ProdProj (projects with productive sector involvement only) indexes, but a new index derived from both was more elucidative.

Figure 2 describes obtained clusters for P.ProdPublProj: P.ProdProj ratio.

Figure 2
Influence of public sector in stimulating interaction with productive sector.

Cluster 3 also presents the high influence of public sector, although not clearly evidenced by other indexes. This new index evaluated simultaneous involvement of public and productive sector in the projects, to enable evaluation of the influence of public sector to promote interaction of the university with the productive sector. Clusters 3 and 4 present high public influence in stimulating TT projects with productive sector. Main managerial, strategic and public policy implications comprise the need for long-term strategies and actions to promote the evolution of the TT process in academic units for a more autonomous partnership, that occur without public incentive.

The institutional innovation system effectiveness construct was also better described one-dimensional by other derived indexes: (i) the amount of registered patents/Amount of R&D Project (Patent_RDproj); (ii) patents, including multidisciplinary patents, with joint ownership with the productive sector/ amount of patents (P.JOPatent); and (iii) Amount of patents registered with involvement of at least one other academic unit (P.MDiscPatent). They evaluated results and innovation system output, quantifying patent generation, and enabled estimating its quality advocated by the current innovation system. One of them, the patent/R&D project (Patent_RDProj) ratio bigger than one (1) called the attention and is presented in Figure 3.

Figure 3
Analysis of the ratio of registered patents X registered research and development projects.

This figure uses the academic units’ global data (total patent amount/total R&D project involving knowledge transfer). Patents could be generated by R&D projects without interaction with the external community (as purely academic thesis, dissertations, and undergraduate courses). Hence, this index was very useful to estimate the patents that are purely academic research projects that do not present knowledge transfer to the productive sector or public institutions.

One researcher affirmed, regarding the patenting initiatives: “there are people that register intellectual property because they obtain points (for career progression): they are not real innovations, nor present commercial potential”.

This is a testimonial about an undesired and opportunistic reaction to the established indicators. Literature refers to the creation of both beneficial and deleterious informal mechanisms as an answer to established indicators and legal dispositions (such as the Bayh-Dole Act). The reasons were not mentioned, but considering the literature, it can have two mechanisms: the engagement of inexperienced academics in patenting results with no commercial value and decline of patent quality even for experienced academics (Czarnitzki, Hussinger, & Schneider, 2011; Grimpe & Fier, 2009). An in-depth analysis showed that undergraduate discipline related patents were also present in the database. This reinforces the need for several actions such as conduction of “competitive intelligence studies” preconized by recently updated Brazilian innovation law (Marco legal da Inovação) (Brasil, 2016).

The indexes considering joint ownership with the productive sector (JOPatent_ProjProdS) and multidisciplinarity represented by patents registered with the involvement of at least one other academic unit (P.MDiscPatent) was elucidative as indicative of the likelihood of alignment with market needs and patents’ commercialization potential. The measures enabled identification of clusters that require attention and investigation regarding the patents without market relevance (Clusters 3 and 4). Patents with joint ownership with the productive sector, especially when maintained by the company, demonstrate the real applicability and effective knowledge and TT. The associated analysis of indexes enabled the identification of a hypothesis regarding the existence of informal mechanisms developed as an answer to established indicators, including deleterious ones, usually unexpected by the national innovation system when laws to promote innovation and technological interaction between universities and the productive sector were defined.

5.3.2. Evaluation of microculture and idiosyncrasies

The microculture and idiosyncrasies were evaluated by multidimensional evaluation, considering cluster composition and comparison.

In the Knowledge transfer by teaching activities cluster (C1), low patent generation and interaction with the productive sector of both Food Science and Technology Institute and Veterinary Medicine was intriguing because of their applied nature, and also because they are related to regionally outstanding productive areas (food and agribusiness sector). The patent data (World Intellectual Property Organization, 2013) shows that related global industry demands new technological knowledge in a global context. This observation claims for the following discussions: (i) these productive sectors are not demanding new technological knowledge in this country, or (ii) the academic units are less inclined to interact with the productive sector. TTO officers affirm that the Food Science and Technology Institute has been very active in the past when the unit’s business incubator was still active. The involvement of undergraduate students in incubated companies and a culture of interaction with companies were observed.

The multidisciplinary nature of these two knowledge areas (food and veterinary) would be expected. For example, the Veterinary unit has no registered intellectual property, although the Biotechnology department (Life Sciences unit) has registered related intellectual property. Hence, the promotion of interaction between Veterinary, Food Science and Technology, and Biotechnology units would be desired, because new technologies related to veterinary knowledge may originate using biotechnology. TTO officers perceive that companies usually guide multidisciplinary projects; thus, it seldom occurs by the initiative of academic units. The general perception is that research groups are closed, preferring to work autonomously rather than interacting with other units. The TTO officers, on various occasions, suggested researchers that they interact with other units to complement their knowledge to better develop the technology. Normally, the perception is that researchers are little receptive to this idea, or have difficulties in establishing this kind of partnership. Due to this aversion to interaction, these researchers end up wasting considerably more time and resources than if they counted on the partnership of another researcher in a related knowledge area, who could more rapidly propose a solution to the problem.

Multidisciplinarity cluster (C2) members present some generation of patents, with emphasis to multidisciplinary patents. The patent generation amount of academic units appears to follow the national tendency. This cluster distinguishes itself from the others by its multi- or interdisciplinary behavior, which is related to improved chances of commercial applicability.

C3 cluster (high generation of patents) stands out in R&D projects and patent generation. The amount of conjoint ownership patent generation with productive sector of the Engineering Academic Unit (11% of registered patents) is lower than the other members of the cluster (Life Sciences 29%; Pharmacy 43%; Chemistry 44%). This observation is important, especially if we consider that there are less pharmaceutical and biotech companies in Brazil than engineering companies. And also is accepted that research in engineering may be more directly applicable to firms than is the case in other disciplines (Bourelos et al., 2012).

It is possible to enquiry the occurrence of at least one of the following reasons: (i) the low applicability of conducted R&D projects; (ii) the low interest of local industry for technological innovation; (iii) or low conscience of local industries about university as a source of knowledge or technology (OI practices and market for know-how and technology absence or incipiency).

These cluster members present high interaction with the productive sector, but apparently by the influence of the public sector inducing interaction with the productive sector: interaction with the productive sector is low where public sector involvement is absent. Long-term policies and strategies must deal with the evolution for a more autonomous relation, which may occur even when the public incentive is absent.

Another discussion regards lower multidisciplinarity (although higher when considering joint ownership with the productive sector), which calls attention to the possible existence of a sense of self-sufficiency, because of the multidisciplinary, applied nature of their discipline.

The Extension cluster (C4) presents high knowledge transfer, but with an inexpressive number of registered patents. The applied nature of knowledge produced by Dentistry course would be expected to generate research fields useful for the development of dental products. Its extension activities essentially comprise providing services to the community, using established techniques. This faculty has a strong patient-focused vision, which could be also useful in developing applied products, even when they are not very innovative in terms of technology. Such developments could be patented. However, the importance of interacting with other fields of knowledge, as chemistry and engineering, in multidisciplinary projects is highlighted.

The cluster influenced by the public sector (C5) presents an emphasis on R&D projects and apparently has a significant influence of the public sector to establish these projects. Public sector influence is occurring in two ways: (i) specific public departments, such as Sanitation and Energy demand specific knowledge from the university, and (ii) public institutions (or public companies) demand projects that require joint activity with a company from the productive sector. The latter do not include interaction with or promoted by government funding agencies (funding calls). These units have had a significant number of cooperation projects with public departments, although the demanded knowledge, by international scientific standards, is not absolutely state-of-the-art, but necessary for regional technological development.

The low patent generation of some members of this cluster indicates low innovativeness, or a low tendency to protect IP, even with high innovativeness. Specifically, more significant patent registration was expected by the Informatics Institute, because the following related technology categories present significant patent registration in countries as United States, Japan and China: Digital Communication (2^nd in China), Computer technology (3^rd in China, and 4^th in India and Japan), Semiconductors (5^th in Japan), IT methods for management (9^th in India), for example. The Hydraulic Research Institute’s closest related technologies are Environmental Technology and Civil Engineering. The former does not present a significant amount of patent registration in Brazil, but the latter is 8^th in patent registration.

6. Limitations and requirements for PMS improvement

Regarding data used to conduct the analysis, it comprised unstructured TTO databases, and, hence, the analysis reflects these restrictions. Because of the unavailability of data regarding the amount of faculty actually acting as researchers, several indexes were relativized. The following variables are desirable for improved analysis: (i) faculty research engagement rate; (ii) researcher’s productivity per academic unit, (iii) spin-off generation. The range of time for each must be defined for monitoring. Other limitations of the study include the infeasibility to analyze direct causal relationships between each project and patents because there was no structured database connecting projects with generated patents. These connections exist, but will always be incomplete, because projects usually generate results at their end, and patents may be registered after a period of time.

The employed method does not evaluate the influence of modifications of behaviors and research related decisions on time, that deliberate modification in cluster formation. Each analysis will be independent of the previous one. Other methods may be explored to enable easy detection of change.

Another limitation is the use of secondary and formal data (TTO records). The analyzed process and system present complexity beyond formal, contractual, and patent filing data. Unfortunately, regardless the potentiality of a multivariate approach to depicting cultural elements, the data do not enable the consideration of factors as leadership and other intangible values and partners’ localization, for example. Previous studies mention the relevance of geographical location (De Fuentes & Dutrénit, 2016), corporate value (Ab Hamid, 2015), and other possible links with industrial partners (such as informal and human resource links) (Vedovello, 1997). Etzkowitz (1998) also indicates the influence of interaction modes in the way researchers interpret their role (and science’s role), and hence, direct their actions and contributions. Future studies may use other data sources to complement the analysis. For example, coauthorship in bibliographical publications, social media data (Research Gate, Linked In or even Facebook, for example), opinions, and sentiment analysis could enable more informative data to deal with the complexity of the process.

Some may argue that using academic departments as analysis units for TT culture evaluation may have undesired effects, as the avoidance of highly innovative multidisciplinary collaboration. However, faculty hiring processes in Brazilian public universities are based on academic departments, with low mobility, as these are permanent, and research groups and labs function as temporary aggregating units with a common objective. The multidisciplinary-related indexes were considered relevant for analysis and appear to be effective.

7. Final considerations

This paper discusses improvement in TT process management while referring to process monitoring. In spite of regulations created to clarify the role of universities in national technological development and innovation, there are several pieces of evidence of outdated practices, behaviors, and values. Moreover, direct strategy deployment is not viable in the analyzed context, while management may act mainly as a motivator of engagement practices, without authority to compel faculty members.

Notwithstanding methodological limitations, joining quantitative analysis with the perception of TTO officers complemented by external data consideration was useful to gain in-depth knowledge about the process and to identify micro cultures. Informal TT mechanisms are not analyzable using data (or variables) available in common databases. Although efforts to integrate this information are important, to formalize all kinds of TT mechanisms is not necessarily effective. Perceptions of faculty members and TTO staff must be used to complement formal data, incorporating more complex and human interactive analysis procedures, to define specific policies and actions to obtain effective decision making.

This paper describes cluster analysis as a tool to structure the monitoring system. Although individual indexes enabled the analysis of project engagement patterns, similarity patterns between academic units (clustering and its composition analysis) and comparison between the adequacy of patent generation rate, and TT project engagement patterns have shown a more elucidative method. Considering that companies’ participation is essential for the development of market-directed projects, the results enabled discussions about the adequacy of both university and companies’ culture from specific local, regional, and national industries.

In some knowledge areas, interaction seems to be mainly induced by the public sector (mainly public funding). The creation of proactive mechanisms seems to be necessary to stimulate interaction culture between academic units and the productive sector, evolving from public funding oriented UI interaction to more refined partnerships.

A hypothesis regarding the existence of informal and deleterious mechanisms, developed as an answer to established indicators, was generated. Managers may question the advantage of investing in registration or maintenance of IP (patents) that do not result from interaction with the productive sector, without at least conducting IP related competitive intelligence studies, due to a lower probability of market alignment.

The literature argues that IP management should be different according to the product life cycle stage. In some stages, more openness is needed, whereas, in others, IP protection plays a major role. Considering this, literature starts to show that Bay-Dole Act-like laws stimuli of patenting activities in universities may hinder the dissemination of useful basic research related knowledge.

To create IP markets and to effectively promote Open Innovation philosophy, universities should not advocate the exacerbated IP protection philosophy associated with the traditional approach of capturing value by the right to exclude other parties. However, a clear understanding of some level of “openness” must be disseminated to effectively facilitate knowledge sharing among stakeholders.

Universities must also consider the risk of patent trolls as side effects of IP markets (Carraz et al., 2014). With the local efficiency view and goal in rising indicators, the real effectiveness in a global sense is absent. The following discussions are needed: Are the licensing partners adequate? Is the exacerbated IP protection philosophy beneficial for local and national industry (especially because this protection occurs at the national level, usually not in other countries)?

Universities and research institutes must implement monitoring systems to enable diagnosis and discussion of prevailing micro-cultures, adjusting institutional and cultural elements. As the obtained clusters are time restricted, their evolution would allow for cultural variation monitoring discussions. During the implementation of the monitoring system, exploring other data mining approaches may be considered with greater potential to elucidate the phenomena. Other multivariate methods, such as factorial analysis and structural equation analysis, may also be useful to evaluate the influence and dependency of factors, for example.

Our analysis proposed a tool for conducting a reflexive discussion about Higher Education, considering the effectiveness of the institutional innovation system in academic units, and its implications for the social and technological development of industry and society.

Appendix A indexes and cluster description.

index Code & Description		p
Project effectiveness	Percentage: Registered know-how transfer project	0.102
	Patent_professor Ratio: patents/ quantity of faculty****	0.000
	Patent_KTproj Ratio: Registered patent amount/Amount of knowledge transfer Project****	0.000
	Patent_RDproj Ratio: Registered patent amount/Amount of R&D Project****	0.001
	JOPatent_ProjProdS Ratio: Registered patent with joint ownership with productive sector/Amount of projects with productive sector***	0.004
	UAPatent_ProjWOProdS Ratio: Amount of patents without joint ownership of productive sector/amount of projects that do not involve productive Sector***	0.004
Commercialization potential	P.JOPatent Percentage: patents (including multidisciplinary patents) with joint ownership with productive sector/ amount of patents***	0.02
	P.UAPatent Percentage: patents (including multidisciplinary patents) without joint ownership of productive sector/ amount of patents***	0.02
	P.MDiscPatent Percentage: Amount of patents registered with involvement of at least one other academic unit****	0.01
Project engagement profile	Proj_Professor Ratio: projects/ quantity of faculty*	0.034
	TeacProj_Professor Ratio: teaching projects/ quantity of faculty	0.061
	ExtProj_Professor Ratio: extension projects/ quantity of faculty	0.082
	RDProj_Professor Ratio: Research and development projects/ quantity of faculty*	0.026
	TeacProj Percentage: Teaching projects/ total amount of knowledge transfer projects	0.085
	ExtProj Percentage: Extension projects/ total amount of knowledge transfer projects	0.630
	RDProj Percentage: Research and Development projects/ total amount of knowledge transfer projects*	0.013
	P.ServProj Percentage: Service providing projects/ total amount of knowledge transfer projects*	0.023
Partnership profile	P.PubProjT Percentage: Projects with public sector/ total amount of knowledge transfer projects*****	0.001
	P.ProdProjT Percentage: Projects with productive sector/ total amount of knowledge transfer projects*	0.021
	P.ProdPublProj Percentage: Projects with both public and productive sector/ total amount of knowledge transfer projects***	0.02
	P.ProdProj Percentage: Projects with productive sector without public sector/ total amount of knowledge transfer projects	0.993
	P.AcProj Percentage: Academic interaction projects/ total amount of knowledge transfer projects***	0.005

Significance in Kruskal Wallis independent test: *significant at 0.05;

**

significant at 0.01;

***

significant at 0.005;

****

significant at 0.001

Acknowledgements

Authors would like to acknowledge financial support by CNPq (Proc. CNPq. No. 460785/2014-1), CAPES, FAPERGS (Proc. 2842-25.51/12-0 and FPE no. 161/2011). Authors would like to thank the institutional support offered by UFRGS’ TTO and Science Park Director, TTO officers, and the interviewed researchers, and the two anonymous referees for insightful and constructive comments, which helped improve this manuscript.

• How to cite this article: Gusberti, T. D. H., & Dewes, M. F. (2017). Impact evaluation for University-Business Cooperation and Technology Transfer in higher education systems: cluster analysis. Production, 27(spe), e20162203. http://dx.doi.org/10.1590/0103-6513.220316

References

• Ab Hamid, M. R. B. (2015). Value-based performance excellence model for higher education institutions. Quality & Quantity, 49(5), 1919-1944. http://dx.doi.org/10.1007/s11135-014-0082-z
  » http://dx.doi.org/10.1007/s11135-014-0082-z
• Ackermann, F., & Eden, C. (2010). Strategic options development. In M. Reynolds & S. Holwell (Eds.), Systems approaches to managing change: a practical guide (pp. 135-190). London: Springer London.
• Ackoff, R. L. (1979). The future of operational research is past. The Journal of the Operational Research Society, 30(2), 93-104. http://dx.doi.org/10.1057/jors.1979.22
  » http://dx.doi.org/10.1057/jors.1979.22
• Ackoff, R. L. (1981). The art and science of mess management. Interfaces, 11(1), 20-26. http://dx.doi.org/10.1287/inte.11.1.20
  » http://dx.doi.org/10.1287/inte.11.1.20
• Adams, N. M., Blunt, G., Hand, D. J., & Kelly, M. G. (2000). Data mining for fun and profit. Statistical Science, 15(2), 111-126. http://dx.doi.org/10.1214/ss/1009212753
  » http://dx.doi.org/10.1214/ss/1009212753
• Agrawal, A., & Henderson, R. (2002). Putting patents in context: exploring knowledge transfer from MIT. Management Science, 48(1), 44-60. http://dx.doi.org/10.1287/mnsc.48.1.44.14279
  » http://dx.doi.org/10.1287/mnsc.48.1.44.14279
• Argyres, N., & Liebeskind, J. P. (1998). Privatizing the intellectual commons: universities and the commercialization of biotechnology. Journal of Economic Behavior & Organization, 35(4), 427-454. http://dx.doi.org/10.1016/S0167-2681(98)00049-3
  » http://dx.doi.org/10.1016/S0167-2681(98)00049-3
• Azevedo, C. S. (1992). Planejamento e gerência no enfoque estrategico-situacional de Carlos Matus. Cadernos de Saude Publica, 8(2), 129-133. http://dx.doi.org/10.1590/S0102-311X1992000200003
  » http://dx.doi.org/10.1590/S0102-311X1992000200003
• Barreyro, G. B., & Rothen, J. C. (2008). Para uma história da avaliação da educação superior brasileira: análise dos documentos do PARU, CNRES, GERES E PAIUB. Avaliação: Revista da Avaliação da Educação Superior, 13(1), 131-152. http://dx.doi.org/10.1590/S1414-40772008000100008
  » http://dx.doi.org/10.1590/S1414-40772008000100008
• Bourelos, E., Magnusson, M., & Mckelvey, M. (2012). Investigating the complexity facing academic entrepreneurs in science and engineering: the complementarities of research performance, networks and support structures in commercialisation. Cambridge Journal of Economics, 36(3), 751-780. http://dx.doi.org/10.1093/cje/bes014
  » http://dx.doi.org/10.1093/cje/bes014
• Bozeman, B. (2000). Technology transfer and public policy: a review of research and theory. Research Policy, 29(4–5), 627-655. http://dx.doi.org/10.1016/S0048-7333(99)00093-1
  » http://dx.doi.org/10.1016/S0048-7333(99)00093-1
• Brasil. (2016, Janeiro 12). Dispõe sobre estímulos ao desenvolvimento científico, à pesquisa, à capacitação científica e tecnológica e à inovação e altera a Lei nº 10.973, de 2 de dezembro de 2004, a Lei nº 6.815, de 19 de agosto de 1980, a Lei nº 8.666, de 21 de junho de 1993, a Lei nº 12.462, de 4 de agosto de 2011, a Lei nº 8.745, de 9 de dezembro de 1993, a Lei nº 8.958, de 20 de dezembro de 1994, a Lei nº 8.010, de 29 de março de 1990, a Lei nº 8.032, de 12 de abril de 1990, e a Lei nº 12.772, de 28 de dezembro de 2012, nos termos da Emenda Constitucional nº 85, de 26 de fevereiro de 2015 (Lei nº 13.243, de 11 de janeiro de 2016) Diário Oficial da República Federativa do Brasil.
• Breschi, S., & Malerba, F. (1997). Sectorial innovation systems: technological regimes, Schumpeterian dynamics, and spatial boundaries. In C. Edquist (Ed.), Systems of innovation: technologies, institutions and organizations (pp. 130-156). London: Pinter.
• Bubela, T. M., & Caulfield, T. (2010). Role and reality: technology transfer at Canadian universities. Trends in Biotechnology, 28(9), 447-451. PMid:20598388. http://dx.doi.org/10.1016/j.tibtech.2010.06.002
  » http://dx.doi.org/10.1016/j.tibtech.2010.06.002
• Burbidge, J. (1995). Back to production management. Manufacturing Engineer, 74(2), 66-71. http://dx.doi.org/10.1049/me:19950208
  » http://dx.doi.org/10.1049/me:19950208
• Byström, K., & Järvelin, K. (1995). Task complexity affects information seeking and use. Information Processing & Management, 31(2), 191-213. http://dx.doi.org/10.1016/0306-4573(95)80035-R
  » http://dx.doi.org/10.1016/0306-4573(95)80035-R
• Carraz, R., Nakayama, I., & Harayama, Y. (2014). Openness, open innovation à la Chesbrough and intellectual property rights. In T. Madiès, D. Guellec, & J. C. Prager (Eds.), Patent markets in the global knowledge economy theory, empirics and public policy implications: theory, empirics and public policyimplications (pp. 209-234). Cambridge: Cambridge University Press.
• Checkland, P. (2000). Soft systems methodology: a thirty year retrospective. Systems Research and Behavioral Science, 17(S1), S11-S58. http://dx.doi.org/10.1002/1099-1743(200011)17:1+<::AID-SRES374>3.0.CO;2-O
  » http://dx.doi.org/10.1002/1099-1743(200011)17:1+<::AID-SRES374>3.0.CO;2-O
• Chiesa, V., & Frattini, F. (2009). Evaluation and performance measurement of research and development: Techniques and Perspectives for multi-level analysis Cheltenham: Edward Elgar Publishing.
• Coriat, B., & Weinstein, O. (2002). Organizations, firms and institutions in the generation of innovation. Research Policy, 31(2), 273-290. http://dx.doi.org/10.1016/S0048-7333(01)00141-X
  » http://dx.doi.org/10.1016/S0048-7333(01)00141-X
• Czarnitzki, D., Hussinger, K., & Schneider, C. (2011). Commercializing academic research: the quality of faculty patenting. Industrial and Corporate Change, 20(5), 1403-1437. http://dx.doi.org/10.1093/icc/dtr034
  » http://dx.doi.org/10.1093/icc/dtr034
• D’Este, P., & Perkmann, M. (2011). Why do academics engage with industry? The entrepreneurial university and individual motivations. The Journal of Technology Transfer, 36(3), 316-339. http://dx.doi.org/10.1007/s10961-010-9153-z
  » http://dx.doi.org/10.1007/s10961-010-9153-z
• De Fuentes, C., & Dutrénit, G. (2016). Geographic proximity and university-industry interaction: the case of Mexico. The Journal of Technology Transfer, 41(2), 329-348. http://dx.doi.org/10.1007/s10961-014-9364-9
  » http://dx.doi.org/10.1007/s10961-014-9364-9
• Etzkowitz, H. (1998). The norms of entrepreneurial science: cognitive effects of the new university – industry linkages. Research Policy, 27(8), 823-833. https://doi.org/10.1016/S0048-7333(98)00093-6
  » https://doi.org/10.1016/S0048-7333(98)00093-6
• Etzkowitz, H., & Leydesdorff, L. (2000). The dynamics of innovation: from National Systems and “‘ Mode 2 ’” to a Triple Helix of university – industry – government relations. Research Policy, 29(2), 109-123. https://doi.org/10.1016/S0048-7333(99)00055-4
  » https://doi.org/10.1016/S0048-7333(99)00055-4
• Fortis, M. A. (2010). Rumo à pós-modernidade em políticas públicas: a epistemologia situacional de Carlos Matus. RAE-eletrônica, 9(2). http://dx.doi.org/10.1590/S1676-56482010000200009
  » http://dx.doi.org/10.1590/S1676-56482010000200009
• Grimaldi, R., Kenney, M., Siegel, D. S., & Wright, M. (2011). 30 years after Bayh–Dole: Reassessing academic entrepreneurship. Research Policy, 40(8), 1045-1057. http://dx.doi.org/10.1016/j.respol.2011.04.005
  » http://dx.doi.org/10.1016/j.respol.2011.04.005
• Grimpe, C., & Fier, H. (2009). Informal university technology transfer: a comparison between the United States and Germany. The Journal of Technology Transfer, 35(6), 637-650. http://dx.doi.org/10.1007/s10961-009-9140-4
  » http://dx.doi.org/10.1007/s10961-009-9140-4
• Guellec, D., & Ménière, Y. (2014). Markets for patents: actors, workings and recent trends. In T. Madiès, D. Guellec, & J. C. Prager (Eds.), Patent markets in the global knowledge economy theory, empirics and public policy implications: theory, empirics and public policyImplications (pp. 9-32). Cambridge: Cambridge University Press.
• Hiatt, S. R., Sine, W. D., & Tolbert, P. S. (2009). From pabst to Pepsi: the deinstitutionalization of social practices and the creation of entrepreneurial opportunities. Administrative Science Quarterly, 54(4), 635-667. http://dx.doi.org/10.2189/asqu.2009.54.4.635
  » http://dx.doi.org/10.2189/asqu.2009.54.4.635
• Hood, C. (1995). The “New Public Management” in the 1980s: variations on a theme. Accounting, Organizations and Society, 20(2/3), 93-109. http://dx.doi.org/10.1016/0361-3682(93)E0001-W
  » http://dx.doi.org/10.1016/0361-3682(93)E0001-W
• Keller, T., & Tergan, S.-O. (2005). Visualizing knowledge and information: an introduction. In S.-O. Tergan & T. Keller. Knowledge and information visualization Lecture Notes in Computer Science 3426 (pp. 1-23). New York: Springer-Verlag Berlin Heidelberg.
• Kerssens-van Drongelen, I. C., & Bilderbeek, J. (1999). R&D performance measurement: more than choosing a set of metrics. R & D Management, 29(1), 35-46. http://dx.doi.org/10.1111/1467-9310.00115
  » http://dx.doi.org/10.1111/1467-9310.00115
• Kerssens-van Drongelen, I. C., & Cook, A. (1997). Design principles for the development of measurement systems for research and development processes. R & D Management, 27(4), 345-357. http://dx.doi.org/10.1111/1467-9310.00070
  » http://dx.doi.org/10.1111/1467-9310.00070
• Lehtinen, J., & Ahola, T. (2010). Is performance measurement suitable for an extended enterprise? International Journal of Operations & Production Management, 30(2), 181-204. http://dx.doi.org/10.1108/01443571011018707
  » http://dx.doi.org/10.1108/01443571011018707
• Leitão, S. P. (1987). Indicadores de desempenho na universidade: uma avaliação. Revista de Administração Pública, 21(2), 55-72.
• Lurie, N. H., & Mason, C. H. (2007). Visual representation: implications. Journal of Marketing, 71, 160-177. http://dx.doi.org/10.1509/jmkg.71.1.160
  » http://dx.doi.org/10.1509/jmkg.71.1.160
• Matus, C. (2006). O Plano como Aposta. In J. Giacomoni & J. L. Pagnussat (Eds.), Planejamento e Orçamento governamental (Vol. 1, pp. 115-144). Brasília: ENAP - Escola Nacional de Adminsitração Pública.
• Meisel, S., & Mattfeld, D. (2010). Synergies of operations research and data mining. European Journal of Operational Research, 206(1), 1-10. http://dx.doi.org/10.1016/j.ejor.2009.10.017
  » http://dx.doi.org/10.1016/j.ejor.2009.10.017
• Meyer, J. W., & Rowan, B. (1977). Institutionalized organizations: formal structure as myth and ceremony. American Journal of Sociology, 83(2), 340-363. http://dx.doi.org/10.1086/226550
  » http://dx.doi.org/10.1086/226550
• Meyer, J. W., & Scott, W. R. (1983). Institutionalization and the rationality of formal organizational structure. In J. W. Meyer (Ed.), Organizational environments: ritual and rationality (pp. 261-282). Beverly Hills: SAGE.
• Mintzberg, H., & Lampel, J. (1999). Reflecting on the strategy process. Sloan Management Review, 21-30.
• Nārāyaṇan, V. K., & O’Connor, G. C. (2010). Encyclopedia of Technology and Innovation Management Hoboken: John Wiley & Sons.
• Niosi, J. (1999). Fourth-generation R&D: from linear models to flexible innovation. Journal of Business Research, 45, 111-117.
• O’Keefe, R. M., & Preece, A. D. (1996). The development, validation and implementation of knowledge-based systems. European Journal of Operational Research, 92(3), 458-473. http://dx.doi.org/10.1016/0377-2217(96)00005-7
  » http://dx.doi.org/10.1016/0377-2217(96)00005-7
• Perkmann, M., & Walsh, K. (2009). The two faces of collaboration: impacts of university-industry relations on public research. Industrial and Corporate Change, 18(6), 1033-1065. http://dx.doi.org/10.1093/icc/dtp015
  » http://dx.doi.org/10.1093/icc/dtp015
• Phusavat, K., Anussornnitisarn, P., Helo, P., & Dwight, R. (2009). Performance measurement: roles and challenges. Industrial Management & Data Systems, 109(5), 646-664. http://dx.doi.org/10.1108/02635570910957632
  » http://dx.doi.org/10.1108/02635570910957632
• Pidd, M. (2008). Tools for thinking: modelling in management science (2nd ed.). Chichester: John Wiley & Sons.
• Pidd, M. (2010). Why modelling and model use matter. The Journal of the Operational Research Society, 61(1), 14-24. http://dx.doi.org/10.1057/jors.2009.141
  » http://dx.doi.org/10.1057/jors.2009.141
• Pillai, A. S., Joshi, A., & Rao, K. S. (2002). Performance measurement of R&D projects in a multi-project, concurrent engineering environment. International Journal of Project Management, 20(2), 165-177. https://doi.org/10.1016/S0263-7863(00)00056-9
  » https://doi.org/10.1016/S0263-7863(00)00056-9
• Power, D. J. (2008). Decision support systems: a historical overview. In D. J. Power. Handbook on decision support systems (International Handbooks Information System, pp. 121-140). New York: Springer Berlin Heidelberg. http://dx.doi.org/10.1007/978-3-540-48713-5_7
  » International Handbooks Information System» http://dx.doi.org/10.1007/978-3-540-48713-5_7
• Quinn, R. E., & Rohrbaugh, J. (1983). A spatial model of effectiveness criteria: towards a competing values approach to organizational analysis. Management Science, 29(3), 363-377. http://dx.doi.org/10.1287/mnsc.29.3.363
  » http://dx.doi.org/10.1287/mnsc.29.3.363
• Rousseau, D. M. (2006). Is there such a thing as “evidence- based management”? Academy of Management Review, 31(2), 256-269. http://dx.doi.org/10.5465/AMR.2006.20208679
  » http://dx.doi.org/10.5465/AMR.2006.20208679
• Scott, W. R. (1995). Institutions and Organizations Thousand Oaks: SAGE.
• Segenreich, S. C. D. (2005). O PDI como referente para avaliação de instituições de educação superior: lições de uma experiência. Ensaio. Avaliação e Políticas Públicas em Educação, 13(27), 149-167.
• Selden, S. C., & Sowa, J. E. (2004). Testing a multi-dimensional model of organizational performance: Prospects and problems. Journal of Public Administration: Research and Theory, 14(3), 395-416. http://dx.doi.org/10.1093/jopart/muh025
  » http://dx.doi.org/10.1093/jopart/muh025
• Shane, S. A. (2004). Academic Entrepreneurship: University Spinoffs and Wealth Creation Cheltenham: Edward Elgar Publishing.
• Simon, H. A. (1979). Rational decision making in business organizations. The American Economic Review, 69(4), 493-513.
• Simon, H. A.(1997). Administrative behavior (4th ed.). New York: Free Press.
• Taticchi, P., Tonelli, F., & Cagnazzo, L. (2010). Performance measurement and management: a literature review and a research agenda. Group, 14(1), 4-18.
• Tergan, S.-O., Keller, T., & Burkhard, R. (2006). Integrating knowledge and information: digital concept maps as a bridging technology. Information Visualization, 5(3), 167-174. http://dx.doi.org/10.1057/palgrave.ivs.9500132
  » http://dx.doi.org/10.1057/palgrave.ivs.9500132
• Vedovello, C. (1997). Science parks and university-industry interaction: geographical proximity between the agents as a driving force. Technovation, 17(9), 491-531. http://dx.doi.org/10.1016/S0166-4972(97)00027-8
  » http://dx.doi.org/10.1016/S0166-4972(97)00027-8
• Voss, C. (2005). Paradigms of manufacturing strategy re-visited. International Journal of Operations & Production Management, 25(12), 1223-1227. http://dx.doi.org/10.1108/01443570510633620
  » http://dx.doi.org/10.1108/01443570510633620
• Wieczorek, J., & Hekkert, M. P. (2012). Systemic instruments for systemic innovation problems: a framework for policy makers and innovation scholars. Science & Public Policy, 39(1), 74-87. http://dx.doi.org/10.1093/scipol/scr008
  » http://dx.doi.org/10.1093/scipol/scr008
• World Intellectual Property Organization. WIPO Statistics Datacenter Geneva: WIPO. Retrieved in 5 August 2013, from http://ipstatsdb.wipo.org/ipstatv2/ipstats/patentsSearch
  » http://ipstatsdb.wipo.org/ipstatv2/ipstats/patentsSearch
• Wright, M., Vohora, A., & Lockett, A. (2004). The formation of high-tech university spinouts: the role of joint ventures and venture capital investors. The Journal of Technology Transfer, 29(3/4), 287-310. http://dx.doi.org/10.1023/B:JOTT.0000034124.70363.83
  » http://dx.doi.org/10.1023/B:JOTT.0000034124.70363.83
• Zainko, M. (2008). Avaliação da educação superior no Brasil: processo de construção histórica. Avaliação, Campinas, 13(3), 827-831.

Publication Dates

• Publication in this collection
  2017

History

• Received
  05 Sept 2016
• Accepted
  06 June 2017