Edinburgh Research Explorer Stakeholder influence and local government performance

Although stakeholder theory has been on the academic agenda for more than 30 years, empirical evidence is still scant about stakeholders’ infl uence as determinant factors of local governments’ performance. Evidence from 82 empirical studies on stakeholder infl uence as a source of local government eff ectiveness is assessed in this article. Th e choice of studies was based in a systematic literature review, encompassing articles published in the period from 2007 to 2017 in which stakeholder infl uences are considered determinant factors of local government performance. Th is research contributes to this fi eld of knowledge by synthesizing the pertinent publications and pointing out an agenda for future studies.


INTRODUCTION
The factors that determine performance is an ongoing agenda for public managers. Several scholars have dedicated time and effort to depict where public managers should preferentially invest in making their organizations more successful. There is no consensus about what should be done to prod public organizations toward providing better services to the population. Boyne (2003), Rainey and Steinbauer (1999), and O' Toole and Meier (1999) are examples of such effort. Although models to explain the performance of public organizations find fertile ground in the management literature, in some cases, they have little effect on public service improvement (Arnaboldi, Lapsley, & Steccolini, 2015). This issue is likely to be explained by the variety of interests and powers a given public organization needs to deal with in its daily activities (Bryson, Cunningham, & Lokkesmoe, 2002).
Theories on determinant factors of performance in the public sector has evolved with scholars examining a several cases on human resources (Im, 2009;Im & Lee, 2012), financial resources (Valle & Gomes, 2014), management (Avellaneda, 2009), leadership (Im & Lee, 2012), networks (Bearfield & Bowman, 2017), among others. A significant portion of the theories available focuses on internal rather than external sources of influence upon performance. This fact could be an indication that performance explanation models must include more external factors, as suggested by the contingency theory (Child, 1977). So far, the only explanation available is that actors involved in the process in real life, but not included in the theoretical models, would be able to explain a substantial portion of local government performance.
In order to assess the state of the art of the role of stakeholder influences on local government performance, the investigation reviews scientific papers published in the last ten years, using a systematic literature review protocol (Pagani, Kovaleski, & Resende, 2015). The protocol included databases around the world, guided by Google Scholar. The research expressions are local government, stakeholder, and effectiveness. The following step was conducted by mining the articles looking for those regarded as representations of the mainstream by using impact factors and the citation periodicity, as proposed by Pagani et al. (2015). After filtering, we analyzed the original articles to assess whether they used the three sets of keywords to shed light on whether stakeholder influences can be regarded as determinant factors of local government performance. That said, the main objective of this paper is to outline the state of the art in the field and to point out a future research agenda.
The paper starts with an overview of the theoretical framework. For this purpose, we briefly describe stakeholder theory and its power to explain the sources of influences (external and internal) likely to shape an organization's behavior and performance. After that, we describe how data were collected and analyzed in the systematic literature review to find papers dealing with stakeholder influences, local government, and effectiveness. Discussion about the findings follows the methods section, and we finish the paper with some suggestions for future research.

Stakeholder theory
Since the publication of Freeman's book in 1984, the stakeholder theory has been used as an accessory theory to explain an organization's relationship with its environment. In the beginning, the theory was seen as private sector-based due to the influence of the book itself (Freeman, 1984). Later, and with the help of other scholars, tools to help the identification of stakeholders were devised, and the theory was used to explain other organizations' behavior.
The fundamental concept of who and what can be a stakeholder for a given organization, as proposed by Freeman, is "any group or individual who can affect or is affected by the achievement of the organization's objectives'" (Freeman, 1984, p. 46). In the same vein, Bryson (1988, p. 74) suggested: "[a] stakeholder is defined as any person, group or organization that can place a claim on an organization's attention, resources or output, or is affected by that output. " These are comprehensive concepts, which raise more confusion than clarity. Mitchell, Agle, and Wood (1997) proposed a Venn diagram, including the dimensions of Power, Urgency, and Legitimacy to depict who is likely to be regarded as a stakeholder. This model has helped the identification process to the extent that stakeholder participation in decision or plan-making process is very likely to be regarded as complex and ambiguous. Other scholars have attempted to shed more light on the process of clarifying stakeholder identification and salience. Savage, Nix, Whitehead, and Blair (1991) proposed a framework for to assess stakeholders' influence in 1989 strike at Eastern Airline; Bryson (2004), in the same line, presented several tools to identify stakeholders, by depicting how influential they are and their ways of dealing with those influences.

Issues of local government performance
As a public organization, a local government has as its primary objective to provide public services to the local population. People live in cities and are served by local authorities. Therefore, local government agencies must measure how satisfied citizens are with the services provided, and with the government as a whole. To understand local government performance by spending money wisely, and providing excellent quality services, scholars have dedicated time and effort to devising better ways of assessing the performance of such multifunctional and multi-stakeholder organizations (Bovaird & Loeffler, 2007). Boyne (2003), Rainey and Steinbauer (1999), Avellaneda (2009), among others, have predicted the determinant factors behind public agencies' performance, namely: resources, leadership, management, regulation, and control. To date, the literature is scant to explain other sources of influence, either pushing local government performance forward or to hinder it. Rowley (1997) acknowledged the importance of stakeholders' influences on organizations as participants in the network involving it. Pajunen (2006) provided a framework for understanding stakeholder influence on private organizations' survival.
In the specific case of public organizations, the paper by Rainey and Steinbauer (1999) is one of the few pieces in which stakeholder influences are seen as a determinant factor of government performance.
The complexity could be due to the multi-stakeholder (Bovaird & Loeffler, 2007) and multipurpose (Greenwood & Stewart, 1986;Leach, Stewart, & Walsh, 1994) nature of these organizations, in which the array of stakeholder influences is vast and complex. Each service provided by local government is likely to affect a myriad of stakeholders. These stakeholders are likely to behave differently from those affected by other services.

Stakeholder theory and local government
Stakeholder theory has been used as an accessory theory for the development of knowledge in the local government domain for about 20 years. Bryson (1988) suggested the influence of stakeholders as a critical issue for public and nonprofit organizations' strategic planning.
In the same direction, Gomes (2003) concluded that stakeholders orbit local governments' environment, acting alone or in groups to exert pressure on their decision-making processes. In a paper published in 2010, Gomes, Liddle, and Gomes (2010) proposed a model (five-sided) to depict the sources of influence a given local authority faces from external and internal stakeholders when making decisions. These sources are regulation, collaboration, agenda-setting, legitimization, and control.
In the same line of thinking, Welch (2012) demonstrated stakeholder influence on participation. In another article, Oliveira and Welch (2013, p. 399) tried to explain whether the application of social media technology was related to "the influence exerted by different stakeholders on the organization. " Another example is Aaltonen and Kujala (2010), who tried to understand stakeholder influences on project management decision making. There are several examples of stakeholder influences, but only a few of them deal with impacts on performance. Here we try to fill this theoretical gap.
Despite being on the public management agenda for about 30, there are several gaps in the stakeholder management body of knowledge. There is no doubt about the importance of stakeholder influences on organizations' behavior and performance (Pfeffer & Salancik, 1978). However, there is no certainty about whether stakeholder influences are likely to act as determinant factors of public organizations' performance.

The state of the art of stakeholder theory
The use of a systematic literature review is becoming more and more regular in the business, management, and public administration literature. However, there are only a bunch of references indicating the combination of the keywords "stakeholder theory" and "systematic literature review. " A glance in the Web of Science of database resulted in 16 scientific articles in which these keywords come out. Stakeholder theory comes out as a keyword in 94% (15) out of total. A systematic literature review appears only in 81% (13) of the cases.
In terms of theoretical context, corporate social responsibility is the focus of 50% of the papers. The other eight papers focus on different subjects, such as corporate social performance, entrepreneurial university, green supply chain management, information technology outsourcing, medical device development, responsible leadership, the stakeholder landscape, and supply chain. The papers' publication period ranges from 2011 up to date.
Another example of a systematic literature review involving stakeholder theory is Cintra, Amâncio-Vieira, Suzuki, and Costa (2014), who carried out a bibliometric analysis of Brazilian outlets for the period 2001-2011. Thought the paper suggests a bibliometric analysis, it does not suggest a research agenda. Another example is a paper written by Pedrini and Ferri (2019) in which they present a literature review on stakeholder management in the private sector. Ribeiro (2016) has carried out a bibliometric study focusing on Brazilian outlets. This evidence suggests the opportunity of carrying out a systematic literature review combining stakeholder theory with local government. There is a gap in the literature.

DATA AND METHODS
To carry out the systematic literature review, we used a protocol proposed by Pagani et al. (2015), which considers as inputs the year of publication, the number of citations and the impact factor. Systematic literature reviews are still scarce in the social sciences (Velamuri, Neyer, & Möslein, 2011). However, the technique has been increasingly used in theoretical papers to guarantee greater rigor, robustness, and replicability (Denyer & Tranfield, 2009). This procedure follows systematic steps to achieve this rigor and robustness. Basically, the steps are: i) establish the intent of the research; ii) conduct an exploratory search; iii) define the inclusion and exclusion criteria; iv) search databases, following the criteria defined in step iii; v) analyze the title and abstract in order to filter the results, and finally vi) analyze the remaining papers in full.
The protocol from Pagani et al. (2015), called Methodi Ordinatio, differentiates from others because it also considers some additional steps in order to provide more robustness: the consideration of publication year, number of citations and impact factor. After the verification of these additional elements, the filtering process concludes in an index labeled as "InOrdinatio. " After this procedure, the protocol suggests to rank the papers. In such rank the difference among well and poorly cited papers becomes evident. The researcher is than able to decide how many papers he/she will analyze. The higher the InOrdinatio, the more relevant the article can be considered, taking into account the year, the number of citations, and the impact factor.
Step 1 -This study intends to outline state of the art in stakeholder theory and identify a future research agenda.
Step 2 -An exploratory search was conducted to identify the central databases and the number of results found. This step led to Step 3, in which the databases and other criteria were defined.
Step 3 -Data were gathered come from a literature review carried out in four big databases, namely Web of Science, Science Direct, Emerald, and Sage. As keywords, we used "stakeholder influences, " "effectiveness, " and "local government" for the last ten years. The investigation assessed three main variables, namely year of publication, impact factor, and the number of citations each article received according to the InCites procedures. The preliminary list included articles that received at least one citation in the period 2007-2017. The first search yielded 674 articles.
Step 4 -Using the InCites Journal Citation Reports from Thomson Reuters, the investigation discarded the journals not present in this list. As a result, 397 articles remained. The third filter aimed to remove the articles that received no citations according to the Web of Science, and 195 articles remained. Since some of the articles were considered theoretical, the protocol added the fourth filter, and 139 remained.
Step 5 -Trying to focus on the more significant pieces, calculated by the InOrdinatio (Pagani et al., 2015), the protocol accepted only articles that have received at least one citation per year, which is calculated by the total number of citations ratio to the age of the paper. This calculation resulted in only 82 papers, which composed the final population of articles.
Step 6 -We performed a content analysis of the 82 papers remaining to assess if: • Stakeholder influences were used as some independent variable; • Local government was, in fact, the research locus; and • Some measurement of effectiveness was used as the dependent variable.
The analysis resulted in only ten articles, which have used the three keywords aligned with the objectives of the systematic literature review. The final step of the analysis and discussion of the results are presented in the following section.

To date
The observation started in 2007, as shown in Figure 1. The most significant number of publications occurred in 2014, but the trend line suggests this as a growing subject among scholars.  Figure 2 presents the main themes of the studies. Th e highest concentration is corporate practices, such as corporate social responsibility, collaborative mechanisms, and governance processes. Aft er that, business-based practices, such as strategy, performance management, are also used as the main themes of research. Environmental studies employ stakeholder analysis to a great extent. Aft er the advent of new public governance ideas, stakeholding has become a typical theme to make public organizations more attached to society.

Research locus
Figure 2 presents the research locus of those published articles. Around 30% of the papers focus on the United States, while the United Kingdom is the core of around 9% of them. For Europe, 40% of the papers are dedicated to it, followed by North America (31%), Asia (18%), Oceania (6%), Africa (2%), and multicultural papers (3%).

Research approaches and instruments
Figures 4 and 5 illustrate how data were analyzed in the articles. In terms of the research approach, 50% of the articles used qualitative analysis, 49% quantitative, and 1% both approaches. In terms of data collection, the survey was the most common method used in 33% of the articles, followed by interviews (27%) and case studies (18%).

FIGURE 5 RESEARCH APPROACH EMPLOYED IN THE ARTICLES
qualitative; quantitative qualitative quantitative

Research Aproach
Source: Data analysis.

Analysis of stakeholder infl uences as determinant factors of local government effectiveness
In terms of the keywords used in the literature review, namely stakeholder, local government, and eff ectiveness, the results indicated there is still a research gap in the fi eld. Th is evidence could mean there is no formal knowledge of whether stakeholders are likely to infl uence the local government's performance. As stated in the extant literature, environmental forces exert a strong infl uence on a given organization's performance.
In the universe of 82 articles selected, only 10 (12%) employed the logic of stakeholder influences as determinant factors to local government effectiveness. Four articles used stakeholder influences as independent variables, five had local government as the research focus, and the only one used any concept of effectiveness as the dependent variable. Once again, one can identify a research gap regarding determinant factors of public organizations' performance. Below we present an analysis of the ten articles.

Paper 1:
Carmeli (2008, p. 984) published a paper in Administration & Society in order to assess "the fiscal conditions of local governments in Israel and potential remedies for dealing with this ongoing problem. " The research was carried out using data on local governments' finances. Fiscal distress is used as a measure of negative effectiveness, operationalized through a comparison between revenues and expenditures. Stakeholder influences are seen as hybrid effects on local government finance, which come about in the relationship between local and central governments in Israel. The independent variables are structural, organizational, and hybrid. Structural variables are "local authorities' size, residents' socioeconomic status, and governmental resource allocation. " The organizational variables are "performance evaluation, transparency, and the role of the local government's management. " Finally, hybrid variables are those related to the "relationship between the central and the local governments" (Carmeli, 2008, p. 985). Clear stakeholder influences are seen in the variables governmental resource allocation, transparency, the role of local government's management, and the relationship between the central and the local governments. These influences are operationalized in the form of indicators, and the theoretical contribution is the overview proposed about the three sources likely to explain fiscal distress, namely structural, organizational, and hybrid. Although proposing some normative elements to the theory, the article does not enlarge the stock of knowledge on stakeholder's role in performance.

Paper 2:
Fernandez, Ryu, and Brudney (2008) investigated if politics still matters when it comes to explaining patterns in local government contracting. The article was published in the American Review of Public Administration. Using data from a national survey, they discovered that "contract management capacity, together with various political factors, helps to account for variations in local government contracting" (Fernandez et al., 2008, p. 454). In terms of stakeholder influences, they state that "pressure from citizens for smaller government has a positive effect on local government contracting" (Fernandez et al., 2008, p. 454). This conclusion is further clear evidence that stakeholder influence is directly and positively related to local government effectiveness. In the same line of thinking, they discovered evidence about the influence of public employees, political opponents, and public managers on local government effectiveness. The problem with these findings is that they come from indirect scales, such as agree/disagree, yes/no. In order to explain stakeholder influences, performance indicators should be developed and tested. In this line of thinking, the paper does not enlarge the stock of knowledge on stakeholder's influence as a potential determinant factor to performance.

Paper 3:
Walker and Brewer (2009, p. 425) investigated 135 English local government authorities in order to understand "how can public managers effectively cope with high levels of red tape in ways that sustain or improve levels of organizational performance?" They used data from a survey aiming to explore "informants' perceptions of organization and management (culture, structure, strategy making, and innovation), drivers of service improvement, background variables, and performance" (Walker & Brewer, 2009, p. 431). Stakeholder influences come in their model in the form of external pressure. According to them (Walker & Brewer, 2009, p. 436), "[t]he major source of external pressure in English local government is currently the auditors and inspectors deployed by the central government (there is at least one for each of our sampled services), " whose "decisions can have a substantial and immediate impact on the service. " In this sense, they investigated whether a prospective strategy would be able to mitigate the influence of red tape stakeholders, and the results turned out positive. The paper was published in the journal Administration & Society. As a theoretical contribution, they suggest that "if public organizations adopt a strategy of prospecting, they can mitigate the negative consequences of red tape, whereas reacting simply compounds the problems associated with excessive rules and restrictions" (Walker & Brewer, 2009, p. 442). Even though they did not work with a variable to measure the actions of auditors and inspectors, this seems to be a potential line for assessing control over public organizations.

Paper 4:
Nunkoo and Ramkissoon (2011) developed a model of community support based on social exchange theory. The paper was published in the Journal of Cleaner Production. Data were collected through a self-administered questionnaire from 800 households from the start of 2008 to early 2009 in Mauritius. Stakeholder influences are used as independent variables in the form of residents' support, satisfaction, trust, and perceived level of power. As dependent variables, the research focused on the perceived benefits and perceived costs of tourism. As a theoretical contribution, the paper "delineates the community satisfaction construct in three categories: residents' trust in local institutions, their level of power to influence development and neighborhood conditions and investigates their effects on overall community satisfaction and attitude in a single model" (Nunkoo & Ramkissoon, 2011, p. 982). Once again, stakeholder influences are acknowledged as determinant factors of performance, but not operationalized in terms of measurements.
Paper 5: Qi et al. (2011Qi et al. ( , p. 1251 published in the Journal of Cleaner Production a paper whose aim was "to examine the role of different stakeholders in the diffusion of ISO 14001 using data set at the Chinese provincial level. " Using only the quantitative approach, they used as a dependent variable, "the total number of certificates in a province divided by its GDP. " As independent variables, they used per capita income in each province "to represent the community stakeholders' pressure. " As the type of regulatory stakeholder influence, they used "a composite index named Environmental Treatment Index … which includes the percentage of industrial wastewater meeting the discharge limits, the percentage of industrial smoke and dust emission from boilers meeting the discharge limits, the comprehensive utilization rate of industrial solid waste, innocent treatment rate of urban garbage, repeating utilization rate of industrial water, the proportion of GDP in investment of environmental pollution treatment and the share of environmental protection industry in GDP" (Qi et al., 2011(Qi et al., , p. 1253. They also used foreign investors and customers as other sources of external influences on performance. As a theoretical contribution, they found that foreign customers do have a stronger influence than regulatory and community stakeholders, and this could be regarded as a genuine contribution to theory but limited to only one stakeholder.

Paper 6:
In an article published in the International Review of Administrative Sciences, Welch (2012, p. 95) tried to "understand whether organizations vary in terms of transparency and participation, how the two dimensions are related, and what factors predict them. " He used a survey of local governments in the U.S. Transparency and participation were used as endogenous variables, composed of several items of the survey questionnaire. Various stakeholder influences were channelized, through factor analysis, into three groups, namely Civil Society Influence, City Influence, and State Influence. The article concludes by indicating that civil society influence increases participation, which increases transparency, while city influence increases participation, but reduces transparency. It is essential to notice the relevance of stakeholder influence. Once again, operational stakeholder influence variables are not presented.

Paper 7:
Kimbu and Ngoasong (2013) investigated tourism development in Cameroon by interviewing destination stakeholders. The paper was published in the Annals of Tourism Research. By using stakeholder theory and network analysis, they based the study on analysis of documents and interviews. After analyzing documents issued by the Cameroon Government, interviews were carried out with several stakeholders to assess the best framework for the development of tourism. In terms of stakeholder influences, they concluded that the National Tourism Office is in charge of leading the process of tourism policy formulation. Elected representative stakeholders help the National Tourism Office by participating in policy decisions in order to provide legitimacy to them. The Prime Minister's Office participates in policymaking by ensuring the "effective implementation (power and legitimacy). " Other stakeholders such as government ministries, international donor agencies, and private sector stakeholders provide technical assistance and funding "to be channeled through the NTO to facilitate the effective mobilization of the tourism industry to enhance its development" (Kimbu & Ngoasong, 2013, p. 255). The paper discussed the influences of coordination, legitimacy provision, ensuring implementation, and funding, which were not operationalized as independent variables.
Paper 8: Ballantyne, Lindholm, and Whiteing (2013) presented a study about the United Kingdom, Sweden, Germany, Poland, and Lithuania aiming to examine stakeholder influences on urban freight transport planning, using semi-structured interviews. The paper was published in the Journal of Transport Geography and concluded there is a lack of stakeholder involvement in the local government decision-making processes. As a theoretical contribution, they "establish a framework that can support a mechanism for negotiations to take place to facilitate decision-making that effectively takes the needs of all relevant stakeholders into account in a much more balanced way" (Ballantyne et al., 2013, p. 100). Although a theoretical contribution is established acknowledging the importance of stakeholders' involvement in local government, the authors do not go into detail about the sort of stakeholder influence that would be effective as a determinant factor for improvement of urban freight transport planning.

Paper 9:
Kamal, Hackney, and Ali (2013) published a paper in the International Journal of Information Management, exploring "enterprise application integration (EAI) adoption in the UK local government authorities" (Kamal et al., 2013, p. 61). They used the case study strategy and interviews with local government authorities. Stakeholder influences were identified in several forms, such as stakeholders' pressure and competition as external pressure factors, …stakeholders' support as a support factor, and …central government grants as a financial factor. " As the main theoretical contribution, the article "proposes a systematic process of prioritizing the importance of EAI adoption factors on the adoption lifecycle phases" (Kamal et al., 2013, p. 72). Stakeholders are seen as essential factors in the implementation process, but there is no causality test for warranting the variables as a determinant to performance.

Paper 10:
In a piece published in a journal called Public Health, Marks et al. (2015Marks et al. ( , p. 1194 presented an investigation aimed to "explore the influence of values and context in public health priority-setting in local government in England. " It is a qualitative investigation carried out by using workshops and semi-structured interviews with professionals to assess the effectiveness of several methods for decision support. Stakeholder inputs are regarded as high and medium in several of those methods. Due to the nature of the investigation, the authors did not delve deeper into stakeholder influences. However, some strong evidence is presented, as the following quotation suggests: "Local authorities are democratically accountable to the local population: this was identified by interviewees as a key factor in decision-making" (Marks et al., 2015(Marks et al., , p. 1198). In the same line of thinking, central governments are able to exert some control over local government, because "local government was subject to less central control of local spending decisions" (Marks et al., 2015(Marks et al., , p. 1198. They also found evidence about the influence of democratically elected members on decision-making. This article offers compelling evidence about stakeholder influence upon performance. Box 1 presents a summary of the analysis presented above. As can be seen, stakeholder influences are acknowledged as an essential source of local government effectiveness. Though regarded as sources of public sector performance in the majority of the papers assessed, only in three of them did we find indicators to measure stakeholder influences? Again, this seems to be an essential gap in the literature. This situation is probably due to the difficulties of translating them into feasible variables. For instance, how can one formulate a feasible and useful indicator to measure leadership? Though regarded as very important to explaining performance in the extant literature (Boyne, 2003;Rainey & Steinbauer, 1999), how leadership is likely to be translated into numbers is still a problematic construct.
Another interesting finding of the research is related to the central theme of the articles and where they were published. According to the second column of Box 1, one can note that the power of stakeholder influences over local government effectiveness is seen in several different ways.

Influences are not translated into indicators
Source: Data analysis.

Confronting the Finds with the Literature
Findings presented in this investigation suggest the amount of interest stakeholder theory has been raising among scholars. Furthermore, one can realize that stakeholder theory is not limited to the business literature as it used to be. It is visible now that scholars have been dedicating time and effort to produce normative (Donaldson & Preston, 1995) knowledge for making organizations more effective in both the private and public sectors. As Box 1 suggests, stakeholder's influences impact performance in a great deal, but it is still an ongoing research agenda as only a few articles presented performance indicators. The absence of performance indicators is clear evidence of the amount of work to be done in order to ensure stakeholder influences as a determinant factor to local government performance.
In terms of local government, the number of articles and books have been growing as people realize how important it is for the understanding of public services provision and welfare. As people live in cities, the literature needs to receive contributions on means for making them more livable with better public services. By studying the influence, stakeholders pose upon local government behavior, and performance is a path to the creation of knowledge in this field. The critical issue in studying stakeholder's influence upon local government functioning is complicated due to the number of actors involved as well as in the transmutation of roles. The very same stakeholder is likely to behave in different ways, depending on time and circumstances.
Another issue from the analysis is that stakeholder's influence stems from the internal and external environment. It has technical and institutional characteristics. Power and interest are neither well balanced, nor permanent. Stakeholder's influences studies need to include elements from Sociology, from Political Science, as well as from Business. The force exerted by one stakeholder is likely to create internal and external instability and uncertainty in an organization's internal and external environments. Therefore, stakeholder analysis needs help from Physics, particularly laws of motion, and forcefields. That said, the stock of knowledge available is not at all able to explain local government behavior and performance.

Research agenda
Based on the analysis, we can conclude that there is a vast field for improvement in stakeholder theory. Although well-developed in other areas, very little is known about the salience of stakeholder influences on local government performance. For this reason, we assume that a feasible research agenda would include increasing the number of studies involving local governments and their relationship with stakeholders. As the data indicated, very few studies focused on local governments as organizations. The public management literature requires more studies encompassing how this type of organization can be improved in order to provide better services to the local population.
In the same line of thinking, determinant factors of performance serve as logical links between stakeholder influences and local government behavior. In times of resource shortages, public managers need to devise more effective ways of investing scarce public funds. For instance, how strong is the employees' influence on performance? What is the role of regulatory activities on local government efficiency? Is local social capital an issue for local government performance? Those are questions without proper answers at the moment. Therefore, scholars need to dedicate more time to developing accurate performance indicators to assess stakeholder influences. Operational indicators of stakeholder influences seems to be the crucial task at hand due to the difficulty of finding comprehensive and precise performance indicators through which stakeholder influences are operationalized.
In terms of research methods, very few of the papers used innovative strategies, such as experiment, focus group, ethnography. More investigations should focus on these more interpretive analyses. The same can be said about the research locus. Context would help in making inferences about stakeholder influences on local government performance in a more comprehensive way. For instance, we found only two articles focusing on stakeholder influences on local government effectiveness in South America, but none of them were included in the final analysis (82 articles). It would be worthwhile to have more cross-cultural analysis involving countries from different continents.

CONCLUDING REMARKS
This paper aimed to present evidence from a systematic literature review of stakeholder influences as sources of local government performance. The endeavor was achieved by systematically reviewing the literature looking for articles in which stakeholder influences were found to be effective in local government organizations. The research justification is based on the fact that so far, determinant factors of local government performance are still unknown.
The literature review found 673 articles in the first view. To narrow the scope to state of the art, we removed papers published in un-indexed periodicals. The result was 397 papers considered to have an impact. We also removed papers not quoted every year since publication. Since citations take time to appear, we calculated the number of citation ratios to paper age. The analysis left us with a sample of 82 papers (are presented in Appendix 1). We carried out a content analysis of the papers looking for evidence of stakeholder influences, local government as research locus, and effectiveness as a dependent variable.
Among the 82 papers, only 10 used the three keywords aligned with the objectives of our systematic literature review. Among the ten papers, only three presented indicators of stakeholder influence. From this analysis, we conclude that there is still a massive gap in the literature. Gomes et al. (2010) proposed a five-sided model to depict the sort of influences a stakeholder can exert on local government decision-making. Some of those influences are still not treated in the literature, so we suggest scholars take a closer look at devising better ways to model determinant factors of performance. Our sample only covers local governments so that the further analysis could look at other types of public organizations. We suspect the results would be the same due to the challenge of devising feasible and useful indicators of stakeholder influences.
As with any research protocol, the investigation presented here also has some limitations. One of them is the length of time adopted. The protocol used articles from 2007-2017 in order to assess citations. Naturally, there are more articles published before 2007, but the number of articles was lower than this period. The other limitation is the number of citations. Choosing to analyze only papers with a higher number of citations leaving aside others published in journals not ranked in the JCR would be seen as a bias of the investigation. However, the investigation assumed that a higher number of citations is an indicator of impact upon knowledge creation.
Recalling the statement with which we started the paper, stakeholder influences are ghosts haunting local government managers with their power to help improve performance, as well as to hamper the administration. There are still many mysteries for public organization performance management scholars to unravel. It seems that stakeholder influences are outcasts, and managers look for less ambiguous and dangerous ways to analyze performance. We believe that stakeholder influences are difficult to deal with, and some of them are too abstract to be translated into indicators and figures. However, until we face them, the ghosts will haunt us.