The role of internal auditing in promoting accountability in Higher Education Institutions

El papel de la auditoría interna en la promoción de la accountability en las Instituciones de Educación Superior

Abstract

This paper discusses the link between accountability and internal auditing, particularly analyzing the extent to which the latter contributes to improve the former, in Higher Education Institutions (HEIs). This study applied a questionnaire to the management boards of a sample of HEIs, to empirical analyze the relationship between internal auditing and accountability. The main focus was on internal auditing carried out by the offices or departments in those institutions. The paper contributes to understand how management boards perceive internal auditing to foster transparency and accountability in HEIs, allowing to corroborate that auditing, and particularly internal auditing, favors the institutions’ accountability. In effect, it promotes the principles underlying accountability practices. The information provided in the scope of internal audits is acknowledged as contributing to improve management effectiveness and helping in decision-making. HEIs wanting to create an internal auditing office or to enhance the role of an existing one, should develop this office’s activities so that it becomes an instrument to support accountability and good governance of the organization. The sample consisted of Portuguese public HEIs, universities and polytechnics. Despite a certain international convergence regarding this type of public sector organizations, and regarding their purposes and governance, certain contextual specificities might limit the generalization of the findings for other jurisdictions.

Keywords:
performance assessment; transparency; responsibility; public universities; Portugal

Resumen

Este artículo discute la asociación entre la accountability y la auditoría interna, en particular analizando hasta qué punto esta última contribuye a mejorar la primera, en las Instituciones de Educación Superior (IES). Para analizar empíricamente la relación entre la auditoría interna y la accountability, el estudio se basa en los resultados de una encuesta enviada a los consejos de administración de una muestra de IES, con el enfoque principal en la auditoría interna realizada por las oficinas o departamentos en esas instituciones. El artículo contribuye a comprender cómo los consejos de administración perciben la auditoría interna para fomentar la transparencia y accountability en las IES, lo que permite corroborar que la auditoría, y particularmente la auditoría interna, sirve a la accountability. De hecho, ella promueve los principios subyacentes a las prácticas de accountability. Se reconoce que la información proporcionada en el ámbito de las auditorías internas contribuye a mejorar la efectividad de la gestión y ayuda en la toma de decisiones. Las IES que desean crear una oficina de auditoría interna o mejorar el papel de una existente, deben desarrollar las actividades de esta oficina para que se convierta en un instrumento de apoyo a la accountability y al buen gobierno de la organización. La muestra consistió en IES públicas portuguesas, universidades y politécnicos. A pesar de una cierta convergencia internacional con respecto a este tipo de organizaciones, así como con respecto a sus propósitos y gobernanza, ciertas especificidades contextuales pueden limitar la generalización de los resultados a otras jurisdicciones.

Palabras clave:
evaluación de desempeño; transparencia; responsabilidad; universidades públicas; Portugal

Resumo

Este artigo discute a associação entre accountability e auditoria interna, analisando particularmente até que ponto a última contribui para melhorias na primeira, nas Instituições de Ensino Superior (IES). Para analisar empiricamente a relação entre auditoria interna e accountability, o estudo baseia-se nos resultados de um questionário enviado aos órgãos de gestão de uma amostra de IES, com o foco principal na auditoria interna desenvolvida pelos gabinetes ou departamentos nessas instituições. O artigo contribui para entender como os órgãos de gestão percebem a auditoria interna para promover a transparência e a accountability nas IES, permitindo corroborar que a auditoria e, principalmente, a auditoria interna, serve a accountability. Com efeito, ela promove os princípios subjacentes às práticas de accountability. A informação proporcionada no âmbito das auditorias internas é reconhecida como contribuindo para melhorar a eficácia da gestão e ajudar na tomada de decisões. As IES que desejem criar um gabinete de auditoria interna ou aprimorar o papel de um gabinete já existente, devem desenvolver as atividades desse serviço de modo a que se torne um instrumento para apoiar a accountability e a boa governança da organização. A amostra foi constituída por IES públicas portuguesas, universidades e institutos politécnicos. Apesar de existir uma certa convergência internacional em relação a este tipo de organizações do setor público, bem como em relação aos seus propósitos e governança, certas especificidades contextuais podem limitar a generalização dos resultados desta pesquisa, para outras jurisdições.

Palavras-chave:
avaliação de desempenho; transparência; responsabilidade; universidades públicas; Portugal

1. INTRODUCTION

In the public sector, in the last decades, the New Public Management (NPM) has involved implementing decentralized management, oriented by objectives. Based on criteria of economy, effectiveness and efficiency (3Es), NPM aims essentially to maximize the performance of public managers to benefit citizens, minimizing the consumption of (usually scarce) available resources (Pereira, Alledi, Quelhas, Bonina, Vieira, & Marques, 2017).

Performance evaluation in the public sector has accompanied the trends in the business sector (Hood, 1991; Pereira et al., 2017), with the introduction in the public service management of elementary principles associated with assessing the performance of services provided and the implementation of accountability practices, aiming to improve the quality/price relationship, taking into consideration the expectations of citizens (Sarrico, 2010). The notions of transparency, participation and accountability, gained prominence in the management of the different entities of Public Administration - managers and collaborators must be held responsible for the results achieved with regard to the objectives proposed for a given period, with the introduction of performance indicators that should be constantly monitored (Liu, Cheng, Mingers, Qi, & Meng, 2010; Wall & Martin, 2003). Nowadays there is the decentralization and delegation of powers and competences, based on a paradigm designated as ‘accountability in the public sector’. This concept represents an implicit and simplified idea of transparency and answerability, where public managers assume responsibility towards citizens for creating mechanisms that can increase their trust (Martins, 2012).

Following the reforms stream in Public Administration, to deal with new social and economic challenges, higher education institutions (HEIs) also experienced significant changes. These institutions faced several new problems, requiring innovative solutions and substantial changes in the traditional models of academic management (Martins, 2012; Santiago & Carvalho, 2008).

In Portuguese public HEIs, main changes related to political authorities’ decentralisation of the decision-making process, giving those entities greater autonomy and responsibility, with their management boards becoming compared to the executive boards of private companies (Afonso, 2009; Christopher, 2012; Marques, 2012). Therefore, the State assumed the role of regulator, in that it only controls the most relevant variables, giving greater responsibility to management boards of HEIs for their behavior and action (Silva, 2016), while implementing a set of mechanisms that make them responsible for the management of public resources.

In this context (where HEIs management comes increasingly closer to that of private companies, but not forgetting the aspects of a social and cultural nature to which they are oriented), this paper discusses the relationship between internal auditing and the promotion of accountability in HEIs. Specifically, the study empirically analyses the management boards’ perceptions about the role of auditing in HEIs, particularly internal audits, in promoting accountability, using the setting of the Portuguese public HEIs (universities and polytechnic institutes).

Research on this topic is opportune and brings relevant contributions for current development of internal auditing in HEIs. Recent reforms in the accounting and reporting systems and the increased importance given to the principles of accountability in the good governance of public sector entities, particularly HEIs, have allowed more information to become available. This has led to more responsibility for the results obtained in managing organizational resources and objectives, at the same time ensuring satisfaction of the different stakeholders’ needs or interests. By recognizing the importance of auditing in supporting the accomplishment of organizational objectives and in ensuring the credibility of the information disclosed, a relationship is expected between accountability and auditing, which is important to empirically discuss (International Federation of Accountants & The Chartered Institute of Public Finance & Accountancy [IFAC & CIPFA], 2014).

Empirical studies on the relationship between auditing and accountability are almost inexistent, especially in HEIs. In fact, no empirical studies were found relating these two topics in this sector of activity. Few studies, as below, only dealt separately with the role of internal auditing, internal control or accountability in public sector organizations.

Teixeira (2006) concluded that internal auditing has been consolidating its role within the governance of private business organizations; management boards acknowledge positive effects of internal auditing on risk management, on assisting prevention of losses, and on identifying improvement opportunities in operations management and control. It also allows for more reliable information, contributing for a better efficacy in management and decision-making. Saraiva (2010) analyzed the importance of internal auditing in polytechnics in Portugal, concluding that management boards consider it is mostly important to prevent errors and omissions, rather than to contribute to assessing the efficiency in the procedures to support management boards’ decision-making. Rodrigues (2017) analyzed the activities developed by internal auditing in Brazilian federal universities, evidencing they aim management control, especially focusing on management and operational auditing. Many activities relate to guiding and supporting managers’ decision-making, monitoring the implementation of recommendations of internal/external controlling bodies, and preparing reports, actively promoting transparency. Internal audit units assure that management boards act according to the legislation and to the results regarding the 3Es of budgetary, financial, property and personnel management. Rodrigues, Machado, and Sampaio (2018) developed a study about the profile of internal audit in Brazilian federal universities regarding staff characterization. They found a predominance of post-graduates in accounting, law or management, and an average of six people in the auditing unit, per institution. Almost half of the HEIs did not have enough staff. They finally highlighted a certain risk of lack of resources and effectiveness, given that the structure of the internal auditing unit was for each HEIs to decide.

The current study aims to fulfil the existing gap, adding evidence to the above-referred apparent relationship between auditing practices (either internal or external auditing) and accountability, even if focusing on the perspective of management boards. The relevance of the context of public HEIs is conspicuous, given the resources allocated and their social aims and mission in developing culture and knowledge, which make public accountability practices even more important.

The paper continues with a literature review, where primary concepts concerning accountability and auditing are explained; the relationship between the two is also discussed, with a special focus on HEIs. The next section addresses the methodological issues for the empirical study, followed by the presentation and discussion of the findings. Finally, main conclusions are summarized.

2. LITERATURE REVIEW

2.1 Accountability

Although the concept of accountability has a long historical tradition in political science and accounting, it is currently considered multi-faceted, complex and difficult to delimit (Carvalho, 2009; Lindberg, 2013). The term is also very difficult to translate to some languages (Pinho & Sacramento, 2009). However, it is consensually associated with the principles of accepting responsibility and the obligation to ‘give account’, where the main aim is transparent management, taking performance indicators into consideration.

One of the initial definitions was presented in 1975 by Yuji Ijiri, where he highlighted a perspective more related to accounting, mentioning that accountability is the ‘accounting’s raison d’être’, since accounting records are made by a responsible (accountable agent), who should give account to a certain entity, in order to fulfil the legal obligations that entity is bound by (Nakagawa, Relvas, & Dias, 2007).

Campos (1990) explains that accountability derives from the work by Frederich Mosher published in 1968, where it is presented as a synonym of objective responsibility. It corresponds to a person or entity’s obligation to give account to another, for the performance and results of its actions, being subject to bonuses and incentives for doing so, or penalties and sanctions (Pinho & Sacramento, 2009).

Over time, the concept of accountability has taken on a new dimension both in businesses and in the public sector, where ‘render accounts’ (be accountable) does not only mean an obligation imposed by the legislation in force, but rather an individual responsibility by the entities’ managers to the different stakeholders, for the performance of delegated functions, considering the interests of whoever elects or appoints them, as well as the objectives defined in an action plan.

Aiming to summarize the different definitions and concepts, Rocha (2008, p. 3) refers to accountability as “making public managers permanently responsible for the actions taken in connection with use of the power given them by the society”. Therefore, accountability is an essential element of good governance, holding decision-makers responsible and avoiding the misuse of power (Cameron, 2004), being presented as a contribution used “not only to control public resource management, but also as a way to stimulate economic gains and the efficiency of those resources” (Carvalho, 2009, p. 30). This idea had already been defended by Mulgan (2000), according to whom accountability could be understood as a mechanism for controlling results and dialogue with citizens, increasing the capacity to respond to their needs.

In a broad sense, accountability is realized by citizens, who exercise their voting power to reward and punish, by election or exclusion, their rulers and representatives. But it also takes place in a specific way, in the performance of daily activities and the manner in which public agents and organizations behave (Rocha, 2009).

Despite the significant diversity of definitions of accountability, three common elements arise: objective responsibility for individual performance; ‘render accounts’; and transparency of the actions taken; all considering assessment of performance based on qualitative and quantitative indicators.

From the above, the main distinctive features of accountability can be summarized in the following principles:

• Being held responsible - Lindberg (2013) identifies as the common denominator the responsibility associated with whoever has decision-making power. This person is obliged to give account through measurable information with verifiable indicators. Public managers are held responsible for actions taken, through the attribution of material rewards (of merit) or sanctions (coercion), considering the results obtained and fulfilment of the legal or regulatory norms in force.

• Rendering accounts - Accountability and rendering accounts represent a mutually related binomial, because the latter is a major means of promoting the former, inasmuch as the information provided by management boards allows them to be held responsible for the actions taken in managing public resources (Carvalho, 2009). Consequently, for accountability, it is necessary to assess and disclose information about the actions taken and the resources used, as well as deviations and the respective justification between what is planned and what is actually carried out, as the whole attribution of responsibility derives from making that pertinent and opportune information available. Furthermore, preparing the ‘accounts to be rendered’ means accomplishing with legal requirements and must consider the guidance within legal rules and standards in force, and contain measurable indicators regarding the degree of citizens’ satisfaction with the services provided; quality is seen as a key factor of long-term success (Cameron, 2004; Carvalho, 2009; Lindberg, 2013).

• Transparency - In considering a more wide-ranging concept of accountability, not only the availability of information should be demanded, but also that it should be reliable and accessible, thereby contributing to promoting transparency. The information provided to stakeholders should obey criteria of comprehensibility, relevance, reliability and timeliness (Gonçalves, 2011). ISSAI 20 - Principles of transparency and accountability indicates accountability and transparency as two essential elements of good governance. Transparency is a fundamental element to promote fight against corruption, improved management and responsibility (International Organization of Supreme Audit Institutions [INTOSAI], 2010).

2.2 Auditing

In conceptual terms, auditing derives etymologically from the Latin audire, meaning ‘to hear’ (Crepaldi, 2016). Historically, although evidence has been identified of a similar activity to auditing during the Roman Empire, only with the industrial revolution in Great Britain in the 19th century, auditing emerged as it is understood today, due to the growth in the number of industrial and commercial firms and the consequent need to implement efficient accounting procedures and internal control measures (Costa, 2010).

Considering the changes occurring over the last decades in organizations’ structure, and in the definition of their objectives, which are increasingly vast and wide-ranging, auditing ceased to have “[…] a primary role in detecting fraud, specifically until the beginning of the 19th century”, and came to be responsible “for issuing an opinion about the financial information and/or report prepared by firms, in order to certify if this is or is not credible and reliable regarding the situation it intends to reflect” (Moreira, 2014, p. 89).

However, in an increasingly globalized and competitive economy, where the principles of accountability are of increased relevance in the public sector as in business organizations, the detection and prevention of fraud is not only a responsibility of those dealing with entities’ governance, but also of their own management (Costa, 2010). Therefore, auditing has adapted to the environment where organizations carry out their activity, responding to new social and economic demands, and becoming increasingly an instrument driving organizational change and innovation. While verifying each matter, auditing nowadays not only considers compliance with existing norms, but also the principles of the 3Es (Pinheiro, 2010; Teixeira, 2006), hence enlarging its focus from an essentially financial to a general management scope.

Consequently, it can be said that auditing plays an increasingly strategic role concerning transparency in rendering accounts, takes on social responsibility regarding the information provided to the stakeholders of the different sectors of activity, both public and private, and is increasingly important in promoting the accountability of several organizations.

Kagermann, Kinney, Kutting, and Weber (2008, p. 2) define auditing as a “systematic process of objectively obtaining and assessing evidence” about the current state of an entity, area, financial statement or controlling process, making a “comparison with previously accepted criteria”, and “communicating the results” to the foreseen users. To Crepaldi (2016, p. 3), auditing can be defined as a ‘survey, systematic study and evaluation of transactions procedures, operations and entity’s financial statements’.

Considering that auditing procedures have been obligatory only for some type of entities (e.g. financial institutions, public corporations and government departments), for most organizations, either in the private or in the public sector, the scope of auditing is for each one to decide (Rodrigues, 2017). However, increasing importance has been given to the carrying out of audits considering the classification from the auditor’s standpoint, since the actions performed internally and externally increase the organization’s effectiveness and the reliability of the information it reports (Teixeira, 2012). Therefore, among several classifications, a common distinction is made between internal and external auditing. Nevertheless, it is more difficult to define the role of an internal auditor than that of an external one, although their work is complementary (Morais & Martins, 2013).

The concept of external auditing is mostly associated with validating the information contained in the entity’s financial reporting (compliance with standards), so that the information provided to the different external and internal stakeholders is credible and trustworthy, thereby demonstrating the capacity of the entity to create value and carry out its activities in the future (going concern principle). This type of auditing is carried out by external professionals not belonging to the entity, and are usually occasional audits (Morais & Martins, 2013, p. 22).

As to internal auditing, the Institute of Internal Auditors (Institute of Internal Auditors [IIA], 2012) defines it as an independent activity, of assurance and consultancy, aimed at adding value to and improving an organization’s operations. It is an examination activity provided by an entity to itself (International Federation of Accountants [IFAC], 2014) and assists the organization in achieving its objectives, through a systematic and disciplined approach, to assure and improve the effectiveness of risk management, internal control and good governance processes (IIA, 2012; IFAC, 2014; Leung, Cooper, & Pereira, 2011). “Internal auditing is a management support function, based on a systematic process, using the appropriate techniques, methodologies and tools of auditing” (Morais & Martins, 2013, p. 91).

Internal auditors should hierarchically depend directly from the organizations’ management boards, be considered as elements of senior staff, and act independently and objectively (Costa, 2010).

2.3 Auditing and accountability in HEIs

The report titled International Framework: Good Governance in the Public Sector (IFAC & CIPFA, 2014) refers to the main factors underlying good governance in the public sector, as illustrated in Figure 1. Looking at the subjects considered in the relationship between the various factors represented, the actions described in Section G stand out, as associated with transparency, reporting and auditing, inasmuch as these are essential in promoting organizations’ accountability.

Figure 1
Relationship between the principles underlying good governance in the public sector

In this framework, good governance requires auditing actions that reinforce the credibility of public management, promoting reliable and responsible reporting, transparency and accountability, as well as allowing for increased stakeholder capacity to exercise control of the actions of management boards, making them responsible for these actions. Therefore, the auditors’ role in the public sector should promote a direct link between the transparency and credibility of the organizations’ management bodies, ensuring that strategic objectives are achieved, considering the ethical values established and legal standards in force (Domingos, 2015; IIA, 2012).

Auditing actions represent an a priori reinforcement of the main pillars defined for accountability, since they allow monitoring and controlling of the government bodies’ action. In doing so, auditing strengths: trust in the democratic system; the prevention of corruption and power abuse; improved organizational functioning, effectiveness, responsiveness and learning capacity; and finally, the very legitimacy of management boards (Bovens, 2005, 2006; Speijcken & Bakker, 2011).

Traditionally, public sector auditing (often assuming several designations such as public auditing, government auditing, State auditing or public management auditing), hereafter referred to as ‘public auditing’, is based on the classical concept of auditing, i.e., on the assessment of the legality and regularity of the accounts, grounded on an opinion about past facts, limited to the financial and budgetary control implemented (Dias & Sarrico, 2008; Domingos, 2015). However, as a consequence of the reforms in NPM, this concept has evolved, from being initially directed towards control of public sector accounts (regularity auditing as part of financial auditing), to performance auditing (Costa, Pereira, & Blanco, 2006; INTOSAI, 2007). Therefore, nowadays, auditing assumes a preponderant role, not only in validating financial information, by assessing compliance with standards and rules and thereby verifying financial execution legality and regularity, according to principles of 3Es, but also in controlling non-financial information, namely concerning monitoring public sector entities’ non-financial performance (Dias & Sarrico, 2008; Domingos, 2015; Marques & Almeida, 2004).

Considering the public manager’s obligation to be accountable (accountability duty) for his/her responsibilities, the information and reporting about the performance and outputs of his/her activity are seen as a fundamental means for transparency and to making him/her responsible for actions taken. In this way, higher authorities are able to identify potential errors and irregularities. In developed countries, public sector entities tend to be subject to controlling systems aiming to ensure fulfilment of the principles underlying accountability and limit the power of management boards. These systems include auditing actions that can be carried out by agents from inside or outside the audited entities.

Nevertheless, despite this apparent relationship between auditing and accountability, there seems to be still a long way to run in order auditing to effectively promote the concepts underlying accountability in public sector organizations. There is a need to use methods and techniques focused on the 3Es principles, allowing public sector managers to be held responsible for the results accomplished in their organizations (Carvalho, 2009; Montenegro & Celente, 2016).

As to the HEIs setting, the changes witnessed in their management model, based on the principles of NPM, particularly concerning accountability, strengthened institutions’ autonomy, making compulsory their performance assessment. This model aimed to: reduce their financial dependency on the State budget (public resources), pressuring them to seek alternative sources of finance; increase their income by providing new services; introduce results-oriented management systems, leading to greater operational autonomy for management boards; and consider economic and financial principles based on the criteria of the 3Es. As such, the governance of these entities came to have an influence on the results they can achieve (Marçal‐Grilo, 2003; Marques, 2014).

In general terms, an institution’s autonomy refers to its capacity to make its own choices, considering its objectives and mission. Still, the reforms implemented in HEIs have led to increased control of public expenditure, institutions being held more responsible and more flexibility in their relationship with the Government and society in general. State control has been therefore replaced by autonomy with supervision, with growing social demands regarding the relevance of HEIs, as well as their accountability (Afonso, 2009; Christopher, 2012; Marques, 2012).

Overall, the tendency in recent years to increase HEIs autonomy has also meant increased accountability. This has launched new challenges for the balance between the two notions, implying the definition of management policies allowing HEIs to attain better results in their main areas of action, namely teaching and research (Guri-Rosenblit & Sebkova, 2004). In general, accountability in HEIs takes a variety of forms, such as licensing, accreditation programs, funding allocation mechanisms, and oversight structures (Semyonov & Platanova, 2017).

In this context, auditing actions carried out either by internal departments or external entities, become relevant instruments of internal and external control for verifying, transparently, responsibilities in assessing the performance of management boards, thereby allowing identification of possible deviations from planned objectives, mistakes or irregularities.

3. OBJECTIVES, METHODOLOGY AND DATA

This study makes an empirical analysis of the role of internal auditing and its importance in promoting accountability, using the setting of Portuguese HEIs and taking the management boards’ standpoint. Box 1 displays the objectives (Ob) and research questions (RQ), starting from a general analysis to the specific contribution of internal audits.

Box 1
Objectives and research questions

According to the control framework in the Manual of Procedures of Control of the Portuguese Court of Auditors (Tribunal de Contas, 1999), internal auditors are the main actors in internal auditing in HEIs in Portugal.

To assess organizations’ perceptions, a questionnaire was sent to the management boards of all public HEIs - 14 universities and 20 polytechnic institutes. It was organized into three sets of questions concerning institutional information, the role of auditing in the HEI, and the specific role of internal auditing for the management boards. Closed questions were used, most of which using a Likert scale to answer from 1 to 5 (where 1 corresponds to the lowest value and 5 to the highest), aiming at assessing the degree of importance the respondents attributed to each statement presented (Sampieri, Fernández & Batista, 2006). The sample consisted of 26 institutions, corresponding to 76% of the universe. Data analysis was essentially descriptive, resorting to frequency analyses.

3.1 Sample description

18 out of 26 entities (69%) in the sample were polytechnics. Most of the answers came from entities in central and northern regions of Portugal where HEIs are more concentrated. As to dimension (Figure 2), half of the HEIs in the sample have less than 5,000 students; these are essentially polytechnics (65%). Universities are larger; two out of eight with above 20,000 students. The average number of students in universities was 12,401, while in polytechnic institutes was 5,521.

Figure 2
No. of students per type of institution

4. MAIN FINDINGS

In this section, main findings are presented following the order of main objectives and research questions presented in Box 1.

4.1 Auditing in HEIs

Respondents in the institutions were asked about different types of auditing performed in their institutions, considering underlying objectives, as well as means used, according to the distinction presented in the Auditing and Procedures Manual of the Court of Auditors (Tribunal de Contas, 1999).

As displayed in Figure 3, financial auditing presented the greatest number of responses (n=24), carried out externally in around 36% of the institutions, and internally and externally in 64% (with only one HEI not performing this type of auditing). Therefore, it can be said that assuring financial statements conformity is the HEIs major concern, especially due to the legally imposed obligations regarding rendering the accounts, according to the different norms in force.

Next, with 22 answers (85% of HEIs), is projects or programs auditing, and quality auditing, carried out in 45% of cases by internal and external means simultaneously. The lowest response rate regards to systems auditing, carried out by 18 institutions, corresponding to 69% of the respondent entities.

As for the type of auditing according to HEIs size, except for financial auditing and projects or programs auditing, the use of internal means prevails over external ones in most types of auditing, regardless the size of the organization. In the largest HEIs (n=2 universities), all types of auditing, except quality auditing, were said to be carried out by internal and external means simultaneously.

Figure 3
Types of auditing by INSTITUTION size

As to external audits by the Court of Auditors in the last 5 years, these happened in 54% of the HEIs, of which 36% are universities and 64% polytechnics. Considering the organisational size, measured by the number of students enrolled in the different study cycles, two universities, the largest in the sample, were audited in the 5-year period analyzed, whereas only 38% of HEIs with under 5,000 students were audited by that body in the same period. In the line with the conclusions of Carvalho (2009) for local authorities, the number of HEIs audited by the Court of Auditors in the latter years can be said as has been short. So, HIEs have not been subject to full external control by that oversight body.

Finally, around 58% of institutions (n=15) overall have an internal auditing office/department. There seems to be a direct relationship between the organization size and the existence of this office/department, being more common in larger HEIs; all institutions with over 20,000 students and 80% of those with between 10,000 and 20,000 students, have an internal auditing office.

The next sections present and discuss the perceptions, gathered from the survey, of HEIs management boards regarding the importance of internal auditing and its role in promoting accountability - namely through the pillars of results assessment, transparency and responsibility, in this way contributing to good governance practices in public administration, especially in the higher education subsector.

4.2 Importance of internal auditing

Table 1 displays descriptive statistics of several items regarding the function of internal auditing (Bovens 2006; Costa et al., 2006; IIA, 2012; Rodrigues, 2017; Speijcken & Bakker, 2011).

Considering the HEIs management boards’ perspective, all items show an average importance level of at least 4, i.e., all functions are considered to have a relevant role in internal auditing in HEIs. These results also show that, generally speaking, the concept of auditing is acknowledged as being evolving, inasmuch as high importance is allocated to functions other than examining the legality and conformity of operations carried out. In effect, despite item 1 with a high or very high importance by all responding entities (n=26) - evidencing a predominance of the importance of conformity audits, presenting the general public with a more credible image of the institution (item 13) is also given a very high role by the majority of institutions (n=14). On the other hand, only five items (around 36%) are attributed a very low (item 9) and low (items 10, 12, 13 and 14) role in internal auditing, by one or two polytechnics.

Table 1
Role of internal auditing in HEIs

	Descriptive statics			Frequency
	Average	Standard deviation	N	Very low	Low	Neutral	High	Very high	No answer
[1] Ensure compliance with existing legislation, as well as internal regulation and standards.	4,50	0,51	26				13	13
[2] Advise and support the management bodies.	4,42	0,58	26			1	13	12
[3] Evaluate the optimization of the management of public resources.	4,32	0,56	25			1	15	9	1
[4] Evaluate the performance of the entity taking into account the principles of economy, effectiveness and efficiency.	4,23	0,59	26			2	16	8
[5] Evaluate and monitor the internal control system implemented, as to its effectiveness and efficiency.	4,46	0,58	26			1	12	13
[6] Evaluate the overall management process.	4,24	0,60	25			2	15	8	1
[7] Coordinate and / or support communication and information activities between management bodies and external auditors.	4,00	0,80	26			8	10	8
[8] Identify risk situations and define effective strategies for their control or mitigation.	4,42	0,64	26			2	11	13
[9] Provide consulting services by providing useful and credible information for timely decision-making.	4,00	0,89	26	1		4	14	7
[10] Promote ethics and take a pedagogical role.	4,23	0,82	26		1	3	11	11
[11] Promote performance-based management related to results.	4,12	0,65	26			4	15	7
[12] Promote transparency reporting on accountability.	4,35	0,85	26		2		11	13
[13] Provide a more credible image of the institution on the general public.	4,46	0,71	26		1		11	14
[14] Make visible the internal actions and procedures.	4,27	0,83	26		1	3	10	12

• Source: Elaborated by the authors.

4.3 Internal auditing contribution to accountability principles

The first aspect to be analyzed in this section concerns the role of internal auditing as an instrument of control in rendering accounts about the management of public resources, taking into consideration the results achieved and respective deviations (performance), in the light of the 3Es principles. In order to understand HEIs management bodies’ perception on this, six internal audit objectives that contribute to the premise under analysis (Domingos, 2015; IFAC & CIPFA, 2014; Marques & Almeida, 2004; Pinheiro 2010; Morais & Martins, 2013; Saraiva, 2010) were indicated, as in Figure 4.

Figure 4
Contribution of internal auditing objectives to reinforcing control in assessing the application of public resources, taking into account the principles of the 3ES

Most respondents give high or very high importance to all items, being 4 the level of importance most frequent. The level of agreement with all the objectives is higher than 4.00, varying between 4.00 and 4.27, respectively to objectives 4 and 1. Therefore, assessment the degree of accomplishment of the action plan and possible deviations arising (objective 1) is given high or very high importance by 92% of responding entities (n=24), followed by objectives 3 and 5 (with 88% of HEIs). In turn, identification and mitigation of irregular and fraudulent situations (objective 6), is the objective most frequently showing disagreement, being given a low level of agreement (level 2) by 12% of HEIs (n=3).

The perception on how internal auditing contributes to promoting transparency in information for accountability, was assessed pointing to seven objectives of internal auditing, as in Figure 5. These include actions to monitor and review the information provided, thus contributing to an improvement of the institution’s image vis-à-vis different stakeholders (Carvalho, 2009; IFAC & CIPFA, 2014; IIA, 2012; INTOSAI, 2013) and the credibility and relevance of the information contained in the reporting documents, thereby contributing to an increase in the effectiveness and confidence of communication between HEIs and the stakeholders (IIA, 2012; INTOSAI, 2013; Montenegro & Celente, 2016; Morais & Martins 2013).

Figure 5
Contribution of internal auditing objectives to promoting transparency in reporting on the accomplishment of responsibilities

All the objectives are given high or very high importance by more than 81% of HEIs, although level 4 is the most frequent in all objectives proposed, except for objective 7 (still considered to be very important by 58% of respondents). Like in the principle regarding assessing performance in the use of resources, the average value here is also above 4, varying between 4.00 and 4.46, the former corresponding to objective 5 and the latter to objective 7. Ensuring the credibility, reliability and relevance of documents concerning rendering accounts (objective 1) and assessing whether the internal control system implemented is adequate, effective and efficient (objective 4), gets the same response frequency, with only one institution taking a neutral position; the others (n=25) register high or very high agreement. Contrariwise, only one institution classifies as low (level 2 on the scale) the contribution made by objectives 2, 3, 5 and 7.

It should also be underlined that the neutral position is of greater significance than in the previous principle, given that in all the objectives presented, at least one institution takes this stance, reaching the highest value in objective 6 (n=5, 19% of HEIs). Accordingly, it can be inferred that, the information contained in internal audits reports is understood as mainly destined to the institution’s internal stakeholders, namely management bodies and audited services; the field of action of internal auditing in promoting transparency is limited.

As to the perception on how internal auditing contributes to promoting responsibility, eight objectives of internal auditing were presented to HEIs management bodies, as in Figure 6. These take into account ethics and values, performance and any deviations that may occur taking into account delegated competencies or intended objectives (Gomes, Fernandes, & Carvalho, 2015; Gonçalves, 2011; IFAC & CIPFA, 2014; INTOSAI, 2013) and the contributions to the control and mitigation of the risk inherent in decision making, preventing any abuse of power and other forms of inappropriate behaviour (Domingos, 2015; IFAC & CIPFA, 2014; IIA, 2012; Morais & Martins, 2013).

Figure 6
Contribution of the objectives of internal auditing to holding management bodies responsible

It can be observed that all items are considered of high or very high importance, for at least 69% of HEIs, with the lowest values in objectives 6 and 7. Concerning these last two, they are given neutral importance (level 3 of the scale) in promoting responsibility by 23% and 19% of responding entities, respectively. In turn, the assessment and proposal of improvement actions concerning performance (objective 2), and control and being held responsible for decisions taken and actions implemented (objective 3) are considered by 92% of entities as highly important (level 4 and 5). In the opposite direction, only one entity gives low importance (level 2 on the scale) to objectives 1 and 6, the lowest response rate being in objective 7 (promoting the definition of responsibilities per function in performing each budgetary task, confirmation and/or control), which, even so, reached 88% (n=23). Summarizing, the average response value is between 4.08 and 4.42, the former corresponding to objective 6 and the latter to objectives 2 and 3.

4.4 Internal auditing recommendations and accountability

In order internal auditing to contribute to accomplish with the aforementioned accountability principles, it becomes essential for organisations to assure internal audits recommendations are accomplished, so that to get the best impact on decision-making. Therefore, systems of controlling and monitoring recommendations coming from the internal auditing team, are important to be implemented.

With the purpose of assessing which are the most used mechanisms, the following options, derived from the literature, were presented to the respondents: follow-up audits; indicators related to the degree of accomplishment with the recommendations, and preparation of procedures and/or internal norms.

Figure 7 shows that only 81% of the HEIs in the sample (n=21) answered, of which 85.7% (n=18) agree or totally agree (levels 4 and 5 of the scale) as having implemented the three mechanisms suggested. The preparation of procedures and/or internal norms is the mechanism with the highest answer rate in level 5 (40% of HEIs), specially by polytechnics. Those HEIs that did not answer (n=5), as well as the polytechnic that disagreed as having implemented these mechanisms, did not have internal auditing office/department within their organisational structure.

Figure 7
Controlling and monitoring mechanisms of the implementation of internal audits recommendations

The premise in the literature that internal auditing is one of the bases of good governance in the public sector (IFAC & CIPFA, 2014), makes it important to understand what is the HEIs standpoint about internal auditors’ recommendations in promoting and reinforcing the principles underlying accountability.

Subsequently, intending at assessing about the contribution of internal auditing for decision-making, HEIs management bodies were asked about up to what extent the results from internal audits (reports, opinions, memos, etc.) could affect decision-making (again in a 5-level Likert scale of agreement). Respondents from universities (n=7) strongly agree that information influences decision-making (levels of agreement of 4 and 5, in 57.1% and 42.9% of cases respectively). As to polytechnics (n=18), 11.1% considered this relationship indifferent, whereas the others agree (22.2%) or completely agree (66.7%). This is in line with the conclusions by Teixeira (2006), evidencing that information provided in the scope of internal auditing contributes to improving management effectiveness and helps in decision-making.

Interesting is to note that 93% of the HEIs agreeing or completely agreeing with the contribution of internal audits information to decision-making, have an internal auditing office/department in their organisational structure.

Furthermore, as in Figure 8, university bodies agree or totally agree that implementing recommendations from internal audits: reinforces control in assessing public resources usage, considering the 3Es principles; promotes transparency in information for accountability, and promotes responsibility. Despite 92% of answers (n=17) in polytechnics equals those of universities (agree or totally agree), there are two entities disagreeing or for whom internal audits recommendations are indifferent.

Figure 8
Contribution of recommendations from internal audits

5. CONCLUSION

Auditing is one of the main pillars of good governance in the public sector, and must serve accountability. This happens inasmuch as the obligation of rendering accounts about the management of public resources, considering the public interest, must be carried out transparently, allowing for holding responsibility by the results accomplished. The internal auditor, therefore, assumes a preponderant role in assigning responsibility and monitoring strategic objectives, in supporting decision-making, in controlling and preventing power abuses, and in identifying situations of risk, fraud or error (Domingos, 2015; IIA, 2012).

Nevertheless, for the existence of a complete relationship between auditing and accountability within good governance, there is still a long way to be run in the public sector, given the diversity in the state of implementation of reforms happened across countries and institutions. In effect, these reforms in many jurisdictions have concentrated more on financial information, letting to a second position methods and practices adequate to promote an integral accountability, allowing also for performance assessment according to the 3Es (Carvalho, 2009; IIA, 2012).

Regarding HEIs, and particularly in the Portuguese context, recent reforms have been towards: reinforcing autonomy with commitment to rendering accounts; narrowing the relationship between financing and accomplished results; partnerships with different agents; and internationalization and diversification of financing sources. As public HEIs management approached that of private organizations, a need for auditing (internal and/or external) and performance monitoring has arisen (Santiago & Carvalho, 2008; Silva, 2016), inasmuch as the obligation of rendering accounts transparently, and in an accurate and accessible way, became an essential requisite to promote accountability (Semyonov & Platanova, 2017). Accordingly, internal auditing has expanded its role, namely regarding assuring the verification of the 3Es and assessing organizations’ management performance, as well as risks, therefore becoming an important support to management bodies, reinforcing control and trust inherent to their functions.

Considering the above, this paper analyzed the role of internal auditing to promote accountability in HEIs, taking the management boards standpoint, using a survey to Portuguese public institutions. In the sample, there are relatively less universities than polytechnics, the latter showing more dispersion of answers and overall more pessimism regarding the relationship between internal auditing and accountability.

The type of auditing still prevailing is financial statements auditing, carried out by almost all responding institutions, followed by projects or programs auditing. This is in accordance with the analysis by Carvalho (2009) for local authorities, showing the prevalence of these audits may originate in legal requirements.

External audits by the Court of Auditors happened in the last 5 years in about half of the HEIs, with greater intervention in universities, indicating the non-existence of full external control. Regarding internal auditing, there is an office/department in the organisational structure of almost 60% of respondents, hierarchically depending in most of cases, of the rectorate or presidency. Therefore, internal auditing functions are considered at the highest level of the institutions.

Considering the growing importance of internal auditing for good governance and its role in promoting the principles underlying accountability, the following conclusions are summarized, as perceptions of HEIs management bodies, to answer the research questions.

Concerning the importance in general in HEIs, on average, a high or very high degree of importance is given to several objectives of internal auditing, revealing perception of an evolution towards supporting management and risk control - internal auditing is seen as no longer limited to examining the legality and conformity of the operations performed, despite conformity audits continuing to be predominant.

As for the role of internal auditing to promote accountability, three main pillars were considered. Regarding auditing as a tool for assessing the use of public resources (performance), HEIs find internal auditing highly important to the assessment of the degree of accomplishment of the action plan and possible deviations arising in the application of public resources. About transparency in rendering accounts, the most important objective seen for internal auditing is ensuring compliance with existing legislation, regulation and norms. Improving HEIs image in stakeholders’ eyes seems to be a neutral aspect, which can lead to conclude that the information contained in internal audits reports is perceived as mainly to be used by management bodies and services audited, eventually to improve internal governance, reducing their field of action in promoting transparency. Finally, as to promoting responsibility, HEIs find internal auditing very important to evaluation and proposal of improvement actions regarding HEIs performance, and controlling and attributing responsibility for decisions taken and actions implemented.

Management bodies considered the information provided in the scope of internal audits contributes to improving management effectiveness and helps in decision-making. Nevertheless, for this information to have the desired effects, it is necessary to implement control mechanisms and monitor the recommendations, namely via follow-up audits, and defining indicators of the degree of accomplishment with the recommendations. There is a general agreement that recommendations made by internal auditors contribute to promote the principles underlying accountability; holding management bodies responsible is, however, the most important contribution.

In the light of the main conclusions drawn from this study, the initial statement can be reiterated, i.e., auditing, and particularly internal auditing, is perceived as serving accountability, inasmuch as it is acknowledged that it promotes the principles underlying this multifaceted concept. Consequently, HEIs wanting to create an internal auditing office or to enhance the role of an existing one, should consider the above, so as to develop this office’s activities in order to become an instrument to support accountability and good governance of the organization.

Drawing from the HEIs management boards’ perceptions may be a limitation of this study, inasmuch as it does not capture the impact of internal auditing implementation to promote accountability. However, assessing management’s perception / reflection on the contribution of internal audit to the main pillars of accountability, remains important as a step leading to its better implementation.

REFERENCES

• Afonso, A. J. (2009, May/Aug.). Políticas avaliativas e accountability em educação - subsídios para um debate Iberoamericano. SÍSIFO - Revista de Ciências da Educação, 9, 57-70.
• Bovens, M. (2005). Public accountability In E. Ferlie, L. E. Lynn, Jr., & P. Christopher. (Eds.). The Oxford Handbook of Public Management (pp. 182-208). Oxford, UK: Oxford University Press.
• Bovens, M. (2006). Analysing and assessing public accountability A conceptual framework European Governance Papers - EUROGOV. (C-06-01).
• Cameron, W. (2004). Public accountability: effectiveness, equity, ethics. Australian Journal of Public Administration, 63(4), 59-67.
• Campos, A. M. (1990). Accountability: quando poderemos traduzi-la para o português? Revista de Administração Pública, 24(2), 30-50.
• Carvalho, A. C. (2009). O papel da informação e do controlo externo no processo de accountability: o caso dos municípios portugueses (Doctoral Dissertation). University of Vigo, Spain.
• Montenegro, T. M., & Celente, A. L. I. (2016). The public sector auditing as an accounting instrument - A case study in the Brazilian Navy. Journal of business and projects, 7(3), 29-47.
• Christopher, J. (2012). The adoption of internal audit as a governance control mechanism in Australian public universities - views from the CEOs. Journal of Higher Education Policy and Management, 34(5), 529-541.
• Costa, A. F., Pereira, J. M., & Blanco, S. R. (2006). Auditoria do sector público no contexto da nova gestão pública. Revista de Estudos Politécnicos, III(5/6), 201-225.
• Costa, C. B. (2010) Auditoria financeira: teoria e prática(9. Ed.). Lisboa, Portugal: Editora Rei dos Livros.
• Crepaldi, S. A. (2016). Auditoria contábil: teoria e prática (10. Ed.). São Paulo, SP: Atlas.
• Dias, C. B., & Sarrico, C. (2008). Controlo público dos governos municipais: novos horizontes. economia global e gestão, 13(3), 99-118.
• Domingos, C. (2015). Auditoria, contabilidade e controle interno no setor público (7. Ed.). São Paulo, SP: Atlas .
• Gomes, P. S., Fernandes, M. J., & Carvalho, J. B. (2015, setembro). O processo de harmonização internacional da contabilidade pública em Portugal: A perspetiva de diferentes stakeholders In Proceedings from Congresso dos TOC 20 anos: Uma ambição, um compromisso, um rumo (pp.1-31), Lisboa, Portugal.
• Gonçalves, C. M. (2011). A accountability nos serviços autónomos (Master Thesis). ISCAL, Lisboa, Portugal.
• Guri-Rosenblit, S., & Sebkova, H. (2004). Diversification of higher education systems: patterns, trends and impacts, in Diversification of higher education and the changing role of knowledge and research (pp. 40-69). Paris, France: UNESCO. (UNESCO Forum Occasional Paper Series, 6).
• Hood, C. (1991). A public management for all seasons?, Public administration, 69, 3-19.
• Institute of Internal Auditors. (2012). Supplemental guidance: the role of auditing in public sector governance Flórida, EUA: Author.
• International Federation of Accountants & The Chartered Institute of Public Finance & Accountancy(2014). International framework: good governance in the public sector New York, NY: IFAC; CIPFA.
• International Federation of Accountants(2014). Handbook of international quality control, auditing, review, other assurance, and related services pronouncements(Vol. I). New York, NY: IFAC.
• International Organization of Supreme Audit Institutions(2007). Diretrizes para as normas de controle interno do sector público Bahia, BA: TCE, 13.
• International Organization of Supreme Audit Institutions(2010). ISSAI 20: Principles of transparency and accountability Retrieved from https://www.igf.gov.pt/Referenciaisnormasauditoria/Docs_Axs/Ax3_issai_20_e_.pdf
  » https://www.igf.gov.pt/Referenciaisnormasauditoria/Docs_Axs/Ax3_issai_20_e_.pdf
• International Organization of Supreme Audit Institutions(2013). ISSAI 100: Fundamental principles of public-sector auditing Retrieved from http://www.issai.org/media/69909/issai-100-english.pdf
  » http://www.issai.org/media/69909/issai-100-english.pdf
• Kagermann, H., Kinney W., Kutting, K., & Weber, C.-P. (2008). Internal audit handbook Berlin, German: Springer.
• Leung, P., Cooper, B., & Perera, L. (2011). Accountability structures and management relationships of internal audit: an Australian study. Managerial Auditing Journal, 26(9), 794-816.
• Lindberg, S. (2013). Mapping accountability: core concept and subtypes. International Review of Administrative Sciences, 79(2), 202-226.
• Liu, W. B., Cheng, Z. L., Mingers, J., Qi, L., & Meng, W. (2010). The 3E methodology for developing performance indicators for public sector organizations, Public Money and Management, 30(5), 305-312.
• Marçal‐Grilo, E. (2003). European higher education society. Tertiary Education and Management, 9(1), 3-11.
• Marques, M. C. (2012). Modelos de governo e gestão universitária: uma visão sobre o contexto atual Português, Revista Iberoamericana de Gestión e Contabilidad, X(19), 1-17.
• Marques, M. C. (2014). Os indicadores de gestão nas universidades públicas portuguesas: uma metodologia para análise da sua eficiência (Vol. I). Faro, Portugal: Sílabas & Desafios.
• Marques, M. C., & Almeida, J. M. (2004). Auditoria no sector público: um instrumento para a melhoria da gestão pública. Revista Contabilidade & Finanças, 15(35), 84-95.
• Martins, P. G. B. (2012). As universidades fundacionais e o new public management (Master Thesis). University Institute of Lisbon, Lisboa, Portugal.
• Morais, G., & Martins, I. (2013). Auditoria interna - função e processo (4. Ed.). Lisboa, Portugal: Áreas Editora.
• Moreira, N. (2014). A Auditoria (forense) e a fraude In N. Gonçalves, & C. Pimenta(Eds.), As Facetas da Fraude - Cinco Anos de Crónicas na Visão (pp. 90-92). Lisboa, Portugal: V. N. Famalicão.
• Mulgan, R. (2000). Accountability: an ever expanding concept?Public administration, 78(3), 555-573.
• Nakagawa, M. N., Relvas, T. R., & Dias, J. M., Filho. (2007). Accountability: A razão de ser da contabilidade. REPEC - Revista de Educação e Pesquisa em Contabilidade, 1(3), 83-100.
• Pereira, F. N., Alledi C., Filho, Quelhas, O., Bonina, N., Vieira, J., & Marques, V. (2017). New public management and new public governance: A conceptual analysis comparison. Espacios, 38(7); 6-30.
• Pinheiro, J. L. (2010). Auditoria interna Lisboa: Rei dos Livros.
• Pinho, J. A., & Sacramento, A. R. (2009). Accountability: já podemos traduzi-la para o português? RAP - Revista de Administração Pública, 43(6), 1343-1368.
• Rocha, A. C. (2008, novembro). O Processo orçamentário Brasileiro como instrumento de accountability. In Anais do 3º APG - Encontro de Administração Pública e Governança, Salvador, BA, Brasil.
• Rocha, A. C. (2009, setembro). Accountability na administração pública: a atuação dos Tribunais de Contas. In Anais do 32º Encontro da ANPAD, Rio de Janeiro, RJ, Brasil.
• Rodrigues, R. C. (2017). A atuação da auditoria interna nas Universidades Federais Brasileiras (Master Thesis). Universidade Federal do Ceará, Brasil.
• Rodrigues, R. C., Machado, M. V. V., & Sampaio, T. S. L. (2018, July). O perfil da auditoria interna nas Universidades Federais Brasileiras. In Proceedings from 18º USP International Conference in Accouting
• Sampieri, R. H.; Fernández Collado, C., & Batista Lucio, M. (2006). Metodologia de pesquisa São Paulo, SP: McGraw-Hill.
• Santiago, R., & Carvalho, T. (2008). Academics in a new work environment: the impact of NPM on work conditions. Higher Education Quarterly, 62(3), 204-223.
• Saraiva, E. M. (2010). A auditoria interna em instituições de ensino superior - o caso do ensino público politécnico (Master Thesis). Instituto Politécnico de Viseu, Portugal.
• Sarrico, C. S. (2010). On performance in higher education: towards performance governance. Tertiary Education and Management, 16(2), 145-158.
• Semyonov, D., & Platanova, D. (2017). Accountability of higher education institutions Background paper for UNESCO’s Global Education Monitoring Report 2017/8 ‘Accountability in education: Meeting our commitments’. Paris, France: UNESCO .
• Silva, T. M. C. F. (2016). O impacto da nova gestão pública nas universidades: uma reflexão sobre o normativo brasileiro. In Anais do 16º Congresso USP de Controladoria e Contabilidade São Paulo, Brasil.
• Speijcken, R., & Bakker, W. (2011). The elusive quest for the golden standard: Concepts, policies and practices of accountability in development cooperation. Maastricht, the Netherlands: UNU‐MERIT. (Working Paper Series, 2011-018).
• Teixeira, M. F. (2006). O contributo da auditoria interna para uma gestão eficaz (Master Thesis). Universidade Aberta, Portugal.
• Teixeira, V. M. (2012). Impacto da auditoria interna na externa: ótica do auditor externo (Master Thesis). Universidade de Aveiro, Portugal.
• Tribunal de Contas(1999). Manual de auditoria e procedimentos - Volume I Lisboa, Portugal: TC.
• Wall, A., & Martin, G. (2003). The disclosure of key performance indicators in the public sector. Public Management Review, 5(4), 491-509.

• [Original version]
• The researcher Susana Jorge acknowledges that her contribution to this paper was conducted at the Research Centre in Political Science (CICP), University of Minho, supported by the Portuguese Fundação para a Ciência e a Tecnologiaand the Portuguese Ministry of High Education and Science through national funds (UID/CPO/0758/2019).

Publication Dates

• Publication in this collection
  15 May 2020
• Date of issue
  Mar-Apr 2020

History

• Received
  29 July 2019
• Accepted
  03 Feb 2020