SciELO - Scientific Electronic Library Online

 
vol.16The Future Digital Work Force: Robotic Process Automation (RPA)Erratum: A Mapping Study of Scientific Merit of Papers, which Subject are Web Applications Test Techniques, Considering their Validity Threats author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


JISTEM - Journal of Information Systems and Technology Management

On-line version ISSN 1807-1775

JISTEM J.Inf.Syst. Technol. Manag. vol.16  São Paulo  2019

http://dx.doi.org/10.4301/s1807-1775201916002 

ORIGINAL ARTICLE

The e-Government Adoption in Higher Education in Portual: The Case of ISCSP at Lisbon University

Maria Helena Monteiro1 
http://orcid.org/0000-0002-3782-9707

Ricardo Ramos Pinto1 
http://orcid.org/0000-0001-8347-8293

1Universidade de Lisboa, Lisboa, Portugal.

ABSTRACT

This paper studied the adoption of information technologies in the higher education area in Portugal, analysing the case of a centenary institution (Instituto Superior de Ciências Sociais e Políticas - Universidade de Lisboa) and trying to understand how its leadership managed and organized the ICT capabilities to deliver value to students, teachers and all other stakeholders. Innovative ICT capabilities were introduced within a planned, motivated and controlled environment where leadership styles and management practices always focused on business success. We applied a program transformation model based in good practices of policy research innovation and diffusion models complemented with program management approaches. Through the analysis of the official documentation and data supplied by the administrative services we were able to confirm the existence of a progressive digital transformation of this institution.

KEYWORDS Digital Transformation; E-Government; Adoption and Diffusion Models; Program Management; Critical Success Factors; Higher Education

INTRODUCTION

The Instituto Superior de Ciências Sociais e Políticas (ISCSP) of the Universidade de Lisboa (ULisboa), is a centenarian higher education institution that, like many others, has been joining the ongoing wave of information age technologies in the higher education area. The school was founded in 1906 to cater for the needs of future colonial administrators within African and Asian contexts, its origins were, at the time, closely connected to the Sociedade de Geografia de Lisboa (Lisbon Geographical Society) and to a monarchy that was heading towards its demise that finally occurred in 1910. Following the political ups and downs of the Portuguese 20th century, the Institute evolved throughout the first Portuguese Republic, the autocratic regime of Estado Novo and the age of newfound democracy after the 1974 Revolution, followed by integration in the European Community, in 1985.

ISCSP has grown from a few dozens of students to almost five thousand, doubling the number of students in the last decade, and facing the challenge of catering to a student body that must be prepared to face the global society of the 2st century. This paper will present the program of transformation put in place in ISCSP, to adopt and diffuse the ICT innovative practices in the management processes, in the high education core business activities and in the business support activities. We introduce the drivers for business-digital transformation translated into challenges and opportunities for the school (Christensen & Laegreid, 2007). To make the scope of the transformation clearer we developed an Information Model (Business Processes and Information Systems) that was refined along the entire transformation process, merging business knowledge with ICT knowledge (Ward & Peppard, 2002).

To analyse how ISCSP’s management aligned ICT capabilities with the digital implementation process we followed an adoption and diffusion model based in a subset of organizational factors with deep impact on the success of this business-digital transformation (Berry & Berry, 2007; Rogers, 2003). The text is divided into introduction, context characterization, methodological options, analysis of the e-government evolution at ISCSP; leadership and organization and, in the end, the conclusion, where was possible to identify the main critical factors of the business-digital transformation, crossing the empirical data with the work of other autors (Altameem, Zairi, & Alshawi, 2006; Gil-Garcia & Pardo, 2005; Kerzner & Belack, 2010).

CONTEXT CHARACTERIZATION

In recent years, ISCSP has faced - and still faces - a number of challenges that in turn have constituted opportunities for expansion and growth. ISCSP has moved from an 18th century palace to wide and modern facilities designed to allow the growth of higher learning and scientific research. In 2008 had to face the challenge of rethinking and rebuilding its academic offer, adapting it to the models that follow from the Bologna Agreement. The reconfiguration of the academic offer brought new teaching lines within graduate and post-graduate courses, and, at the same time, a new line of non-graduate specialized training and consulting offers were developed, in reply to external requests and complementing graduate education. The fact that several teachers and administrative staff reached retirement age and ended their working careers, created the opportunity to hire younger people that brought new ideas and competencies. In recent years, ISCSP elected its top management four times, bringing new insights to the governance of the institution. Nevertheless, continuity was always ensured by solid intermediate leadership, engaged with the vitality and continuity of the school. The 2008 financial crisis increased the report obligations to several agencies of central government, such as the ministries dealing with Finance, Higher Education and Social Security, and introduced the need to conform to a large number of new rules, that must be taken into account along the entire activity process. Also, the creation of an independent evaluation agency (A3ES - Agência de Avaliação e Acreditação do Ensino Superior), generated new rules - mainly scientific - and procedures that higher education institutions had to respect. The organizational transformations were profound and the necessity to comply with new and complex business solutions resulted in the introduction of new ICT technologies and solutions. The 2008 crises also imposed budgetary constraints and reductions that affected all sectors of the society, in general, and the Higher Education sector, in particularly, forcing the institutions to gain efficiency to be able to respond to the increase on the demand. The merging, in 2013, of Universidade Técnica de Lisboa, where ISCSP was integrated since 1961, with the Universidade de Lisboa - the two biggest universities in Lisbon - to form the new Universidade de Lisboa, was a huge challenge that generated a wide range of opportunities.

The Business Model (Business Processes and Information Systems)

Analysing the ISCSP business strategy and describing it in a structured way we derived the ISCSP Business Model including business process model and business data model. This strategy identified new business processes and the redefinition of others already existing. For this representation we adopted the arquitectural model for business and information system presented by Ward and Peppard (2002:193). The vision of the business data model and the information systems solutions covering the new requirements for the operational and management processes, were designed to accommodate the innovation strategy required by the ISCSP external and internal environment.

In recent years the institution’s activity is permanently demanding information at different levels: 1) operational execution of activities related to research, teaching, business development and administrative tasks; 2) management activity related to receiving information, understanding situations and decision making; 3) strategic planning within the school, the University and the national academia ecosystem.

The ISCSP’s business model design is described at a high level through processes and information subsystems presented in table 1 and table 2.

Table 1 Macro business process and processes of ISCSP 

Macro Processes Processes
Management Processes Strategic Planning and Decision
Annual Planning and Control
Budgetary Management
Human Resources Management
Quality Management
Institutional Communication
Information System Management
Core Business Processes Research
Graduate Education
Specialized Training and Consultancy
Development and Cooperation
Active Citizenship
Services
Support Processes Administrative Academic Task Management
Issuing of Certificates
Event and Conference Management
Administrative and Human Resources Management
Administrative and Financial Management
ICT Management
Management of Facilities and Security
Logistics and Provision Management

Table 2 ISCSP information systems in 2018 

Information Systems
(BI) Business Inteligence
(FA) Finantial & Administrative Management
(EP) Eletronic Procurement
(HR) Human Resources Information Systems Management
(PM) Project Management
(QM) Quality Management
(CRM) Customer Relations Management
(AISM) Academic Information System Management
(NetPA) Digital Academic Office 2.0
(Moodle) eLearning Platform
(KOHA) Digital Library and Book Management
(CICM) Communication and Image Content Management

METHODOLOGICAL OPTIONS

Both of the authors of this paper, during the considered time frame, have been teaching at ISCSP and, during a very significant part of the time, also had management responsibilities in the area that is the object of this investigation. This long professional and sentimental relation with the institution and its digital transformation gave the authors a detailed knowledge of the context and the process undergone, but also represented a huge challenge to a scientific and objective analysis. The methodological strategy that used was to use the authors knowledge of the research object, combined with the state of the art in this area, to define the structure of a series of tables - presented in the paper - that contained the time series of the relevant variables between 2008 and 2018, which were then discussed with key elements of the administrative structure - typically project leaders - and the final version supplied to them for the insertion of the data. This methodological approach, in the authors opinion, ensures the required objectivity that a scientific investigation requires, without depriving the research of the substantial knowledge that the authors had on the context. Complementary was also considered the official and public documentation available on the subject, like annual activity plans and activity reports.

ANALYSIS OF THE E-GOVERNMENT EVOLUTION AT ISCSP

The strategy of digitalization and e-government is more and more common in public organizations to improve efficiency, effectiveness and quality. It fits the general strategy of Europe 2020 and Portugal’s digital agenda, which assumes the OECD (2003:11) definition of e-government as "the use of information and communications technologies (ICTs), and particularly the Internet, to achieve better government”, consisting in the use of "solutions that exploit 21st century information and communications technologies (ICTs) to achieve public policy goals within a context of changing social, economic and political circumstances” (OECD, 2003:3). E-government provides a platform for multi-channel interaction and multi-service delivery options and is about centralized, yet distributed operations to maximize efficiencies, productivity and service delivery (Esteves & Rhoda, 2008). The digital transformation of operations forces, to optimize government services delivery, constituency participation and governance, the transformation of the relationships of the public sector at an internal and external level,” (Esteves & Rhoda, 2008). In e-government, the government uses information technology and particularly the Internet to support government operations, engage citizens, and provide government services. On the other hand, we may define e-government as a continuous innovation in the delivery of services, citizen participation, and governance through the transformation of external and internal relationships by the use of information technology, especially the Internet and mobile devices. The European Commission defines e-government as "the use of ICT in public administration - combined with organizational change and new skills - to improve public services and democratic processes and to strengthen support to public policies” (COM, 2003:7). Janssen & Estevez (2013:S2) give envidence to the diversity and evolution of the e-government definitions, but also recognize the existence of dominate idea which is the fact that it consists on the "use of ICTs to improve the activities of public sector organizations”. According to the previous definitions, ISCSP is implementing an ICT platform in a phased approach, with electronic services for students, teachers, managers and administrative staff. This platform follows the business model previous described and all processes have been redesigned according the digital innovation of each transformation stage (Ojo, Janowski, & Estevez, Improving Government Enterprise Architecture Practice - Maturity Factor Analysis, 2012).

ISCSP e-Services transformation

Since 2009, as previously stated, ISCSP has undergone a profound transformation in its information systems. Following the best practices of strategic planning and corporate IT architecture process (Perks & Beveridge, 2003:12-17) we reached the solution and application architecture. A platform where selected with the fundamental key services to the academia business. Following the annual business planning process, each year several new e-services (automatic functionalities) were identified for candidates, students, teachers and administrative staff. These e-services were planned (P) at the first moment, after they are designed (D) and after the implementation, they are used by the final actors (A).

For describing these e-services evolution, from their identification and planning phases (P), through the design and construction phases (D) to the final kick-off phase and their continuing use (A), the authors used the tables representation with the e-services identification in lines and the years in columns. In each cell we can see the moment of the requirement recognition, the first year with the P (planning), followed by the year were the design (D) were done and after the kick-off of the e-service (the first A). This representation can be observed in tables 3 t o 6.

Table 3 The functionalities available to students in the ISCSP e-Services. 

Functionalities available to students 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Aggregating Portal NETP@ D A A A A A A A A A A
Digital Secretary 2.0
Enrolments and Registrations P D A A A A A A A A A
Photo and personal data updating P/D A A
Registration for Exams
Appeal Period P/D A A A A A A
Special Period P/D A A A A A A
Grade Improving P/D A A A A A A
Grades Inquiries P/D A A A A A A A A A
Exams Dates Inquiries P/D A A A A A A A A A
Inquiries of MB References P/D A A A A A A A A A
NET Documents P D A A A A
NET Alerts
Payment Dates P/D A A A A A A
Exams Dates P D D D A A
Grades Sheets Availability P D D D A A
E-Learning Portal
Automatic registration in the curricular unities attended P D A A A A A
Student/teacher and student/student interaction P D A A A A A A A
Access to academic contents and study materials P D A A A A A A A
Sending of essays for grading with automatic verification of plagiarism P P

Table 4 The functionalities available to teachers in the ISCSP e-Services. 

Functionalities available to teachers 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Aggregating Portal NETP@ D A A A A A A A A A A
Entry of Grades D A A A A A A A A A A
Curricular Unity File P D A A A A
Digital Summaries P P
Digital Signed Grade Sheet P D/A A
Grade Sheet Status Check P/D/A A
Test verification scheduling P/D/A A
NET Alerts
Deadlines for entry of grades P P P P D D
Exams Dates P P P P D D
E-Learning Portal
Automatic linkage to curricular units P/D A A A A A
Teacher/student and teacher/class interaction P/D A A A A A A A
Availability of academic contents and study materials P/D A A A A A A A
Reception of essays for grading with attached report on plagiarism P P

Table 5 The functionalities available to candidates in the ISCSP e-Services. 

Functionalities available to candidates 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Aggregating Portal NETP@ D A A A A A A A A A A
On-Line Appplication
Master courses P D A A A A A A A A
Doctorates P D A A A A A A A A
Post-Graduations P D A A A A A A A A
Specialized Courses P D A A A A A A A A
Special Contests
Over 23 P D A A A A A A A A A
Course Changes P D A A A A A A A A A
International students P D A A A A

The evolution of the interactions among several administrative services at ISCSP and the academic administrative service and all stakeholders - external and internal - is presented table 6.

Table 6 The functionalities available to administrative services in the ISCSP. 

Functionalities available to Services 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Interface for invoice data sharing with the Financial Area P D A A A A A A
Generation of Statistics for Management Support
Presidency P/D A A A A A A A A A A
Management Council P/D A A A A A A A A A A
Services (Academic, Human Resources, Financial, Mobility) P/D A A A A A A A A A A
Official External Entities (Ministry, A3ES, FCT) P/D A A A A A A A A A A
Grade Sheet Status Check P/D/A A
Massive messaging for students, teachers and other stakeholders P/D/A A A

From the analysis of the tables above it is possible to see that in the beginning, efforts were clearly focused on the back-office processes and grades registry. Progressively there has been a shift towards front-offices processes, such as the interaction with candidates, teachers and students. It may be noticed that the latter functionalities were concerned with the digital interactions of ISCSP with other governmental institutions.

The Use of the ICT platforms

All the key elements of the administrative structure that were inquired had the perception that the number candidates, students and teachers that use the face-to-face front office desks decreases annually, even though the total number of students had a 70% increase, from 2682 in 2008 to 4541 in 2017 (ISCSP, 2008; ISCSP, 2017).

It is clear that the increase in the number of functionalities available in ISCSP e-Services has been followed by a sustained increase of its use, which suggests a high level of adoption of by all types of users.

THE MANAGEMENT AND ORGANIZATION OF THE DIGITAL TRANSFORMATION

We will now focus on the analysis of how the ISCSP’s leaders managed and organized the ICT capabilities to deliver value to students, teachers and all other stakeholders.

The preparation and implementation of this transformation in ISCSP has been defined and promoted along a certain period of time, mixing several internal and external organizational factors as used by several authors in models of adoption and diffusion of organizational innovation (Rogers, 2003; Berry & Berry, 2007). The organizational factors analysed at ISCSP business-digital transformation can be divided in: external factors (competition, political pressure and market pressure); and internal factors (strategy & business dimension, motivation, human capital, quality assurance & quality control, constraints/resources and leadership).

External factors (competition, political pressure and market pressure)

The total number of candidates to the Portuguese higher education system decreased in the recent years, reaching a point where it is inferior to the installed capacity of the institutions. ISCSP as presented a sustained growth of the number of candidates and students, nevertheless, has always kept in mind the risk of an inversion of this tendency, due to the competitive environment that was created.

The implementation of Bologna Agreement, in 2008, and, in 2013, the merging of Universidade Técnica de Lisboa with the Universidade de Lisboa, created the necessity to rethink the academic offer an its alignment with the market and, within the University, with the mission of each Faculty. The regulations issued by A3ES forced compliance with stricter rules and imposed the creation of new process and the reconfiguration of several of the old ones. On the other hand, financial restrictions that have been imposed on the area of Higher Education and namely on ISCSP have been quite harsh, particularly in the human resources area.

Students, future students, organizations and external networks with whom both services and researchers interacted, demand modernization and dematerialization of the process, and are particular receptive to web supported services.

Internal Factors (strategy & business dimension, motivation, human capital, quality assurance & quality control, constraints/resources and leadership)

Conscious that the previously identified external factors and the consistent increase of the number of students, made obvious that it was not possible to pursue the activity with the use of the pre-existing tools for the treatment of information. ISCSP’s strategy included the gradual but urgent, implementation of information systems for the support of financial, human resources and academic areas at the transactional and management levels.

The demand and rigor required by operational tasks, especially in the academic and financial areas, were not compatible with non-integrated automatic tools and tasks developed on paper. Such reality led to problems in daily operation: slowness in the execution of urgent tasks causing long queues of students trying to complete their applications, registration and payment; difficulties in the integration of financial movements resulting from payments that need to be reported to the financial area; slowness in the publication of students’ grades; slowness in the issuing of certificates; etc. Administrative staff was unable to increase productivity and lower the risk of manual errors. This model of service and work also generated a number of complaints from students. The whole management and decision-making process suffered correspondingly as to the productivity of the operational tasks. Conditions were met for the creation of the Business Process Reengineering, and Integration of the Academic and Financial Processes with the help of an innovative ICT Platform. The vision of this interoperability was put in place from 2008 until the present time.

The processes of retirement and mobility of teachers and administrative staff created the opportunity for admitting new collaborators, which, in general terms, were younger, with higher qualifications, with greater knowledge and agility in working with informatics tools, as well as greater willingness to learn from older colleagues. This new wave of human resources enabled the creation of internal capacity to accomplish processes the reengineering of required by implementation of the new ICT Platform.

The demand for the quality of internal and external response to the several stakeholders of this activity was always present, through structures in ISCSP specifically dedicated to quality and auditing. It was well known that the introduction of new automatic solutions would generate profound transformations in operational and governance processes. The rigor of the final results of ISCSP’s handling of management, students, teachers, administrative and other staff, as well as the reporting of data to rectory and ministries led to Criteria for Quality Results Control and Criteria for Quality Assurance on the Phases of the transformation process.

Constraints are quite strong when radical changes are introduced from traditional processes (use of sparse non-integrated informatics tools and a high use of paper - this was the view of ISCSP’ 2008) to models of work supported by digital platforms. By making use of several channels, these platforms transversally integrate operational and management processes to serve management, students, teachers, administrative and other staff, and external contacts and engagements in real time. In this context, constraints and barriers at cultural, organizational and management levels are customary. Financial resources are applied to the new solutions according to annual availability, thus contributing to the definition of phased Implementation Projects. Such Projects are part of a global Transformation Program;

Management and Leadership are distinct but related concepts, and both are needed in transformation processes. ISCSP has always kept the operational management in the hands of the Executive Director, and top leadership in those of the President and Vice-presidents. Intermediate leaders operationalize the projects, they are the projects leaders. The Vice-presidents monitor the progress, the interdependences and risks of the projects, supporting the project leaders. The President establishes strategies, priorities and resources for the transformation program. The President is thus the sponsor of the transformation program. (Ojo, Janowski, & Estevez, Whole-of-Government approach to Information technology strategy management: Building a sustainable collaborative tecnology environement in government , 2011).

Adoption and Implementation Model

This implementation model is the result of the integration of the research in innovation and diffusion models in policy research (Berry & Berry, 2007) and the large experience, available in the ISCSP, in managing Public Administration complex Projects

The general model applied in this research is the following:

Adoption (i,t) = f (external factor 1 (i,t), external factor 2 (i,t), , internal factor 1 (i,t), internal factor 2 (i,t),internal factor n (i,t)

Where:

i - Adoption of the component (i) of the Transformation Program

t - Time period (t) to adopt the (i) component delivered from the Transformation Program

Internal and external factors are the ones mentioned previously.

CONCLUSION

In this paper we have explained our research work on the innovation adoption applied at the business-digital transformation of ISCSP.

Analysing all components of this program implementation, during and extended period of time, and crossing it the state the work of other authors (Altameem, Zairi, & Alshawi, 2006; Heeks, 2008; Gil-Garcia & Pardo, 2005; Kerzner & Belack, 2010) we were able to list a subset factors that, in our opinion, were critical for the implementation of the analysed projects. The point of departure of the ICT capabilities and the HR and Financial resources in the Organization was fully understood, enabling a precise diagnose of the problems and challenges that the institution had to address. The institution was also fully aware of the state of the art of ICT capabilities and solutions in similar organizations (benchmark), as well as the national and international public policies about Government Digital Transformation. The annual activity plans evidence the existence of a digital strategy using a phased implementation framework supported in a proved adoption and diffusion models. It was also possible to identify a carefully preparation of the implementation plans focusing on quick and useful results to the stakeholders (efficiency, efficacy and quality), where the implementation teams were configured according the available resources. The annual activity reports denoted the introduction of new communication processes, as well as, the continuous process of revising and readapting the transformation plan according to the external and internal contexts. Even without inquiring all the stakeholders it is possible to conclude that this digital transformation process changed, step by step, the organizational culture and the internal and external image of ISCSP.

REFERENCES

ALTAMEEM, T.; ZAIRI, M.; ALSHAWI, S. (2006). Critical Success Factors of E-Government: A Proposed Model for E-Government Implementation. Innovations in Information Technology Conference, 1-5. [ Links ]

BERRY, F. S.; BERRY, W. D. (2007 ). Innovation and Diffusion Models in Policy Research. En: P. A. Sabatier (Ed.), Theories of the Policy Process. USA: Westview Press. [ Links ]

CHRISTENSEN, T.; LÆGREID, P. (2007). The whole‐of‐government approach to public sector reform. Public administration review, 67(6), 1059-1066. [ Links ]

ESTEVES, J.; RHODA, J. C. (2008). A Comprehensive framework for the assessment of eGovernment projects. Government Information Quartely 25 - Science Direct , pp. 118-132. [ Links ]

EUROPEAN COMMISSION. (2003). The Role of eGovernment for Europe´s Future. Accessed on the 13th of September of 2018. Avalilable in: Avalilable in: https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2003:0567:FIN:EN:PDFLinks ]

EUROPEAN COMMISSION. (2016). EU eGovernment Action Plan 2016-2020: Accelerating the digital transformation of government. Accessed on the 2nd of September of 2018. Avalilable in: Avalilable in: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52016DC0179&from=PTLinks ]

GIL-GARCÍA, J. R; PARDO, T. A. (2005). E-government success factors: Mapping practical tools to theoretical foundations. Government information quarterly, 22(2), 187-216. [ Links ]

HEEKS, R. (19 de 10 de 2008). Critical Success Factors: Why eGovernment Projects Succeed in eGovernment for Development Information Exchange. Accessed on the 18th of September of 2018, in: Accessed on the 18th of September of 2018, in: http://www.egov4dev.org/success/evaluation/factormodel.shtml . [ Links ]

ISCSP. (2008). Relatório de Actividades e Contas, Lisboa: ISCSP. [ Links ]

ISCSP. (2017). Relatório de Actividades, e Contas, Lisboa: ISCSP. [ Links ]

JANSSEN, M.; ESTEVEZ, E. (2013). Lean government and platform-based governance-Doing more with less. Government Information Quarterly, 30, S1-S8. [ Links ]

KERZNER, H.; BELACK, C. (2010). Managing Complex Projects . USA. Canada: Wiley. [ Links ]

OECD. (2003). The e-Government Imperative. Accessed on the 6nd of October of 2018. Avalilable in: https://www.oecd-ilibrary.org/governance/the-e-government-imperative_9789264101197-enLinks ]

OJO, A.; JANOWSKI, T.; ESTEVEZ, E. (2011). Whole-of-government approach to information technology strategy management: Building a sustainable collaborative technology environment in government. Information Polity, 16(3), 243-260. [ Links ]

OJO, A.; JANOWSKI, T.; ESTEVEZ, E. (2012). Improving Government Enterprise Architecture Practice - Maturity Factor Analysis . 2012 45th Hawaii International Conference on System Sciences (HICSS) (pp. 4260-4269). Hawaii: IEEE Computer Society. [ Links ]

PERKS, C.; BEVERIDGE, T. (2003). Guide to Enterprise IT Architecture. USA: Springer . [ Links ]

ROGERS, E. M. (2003). Diffusion of Innovations (5th edition). New York: Free Press. [ Links ]

WARD, J.; PEPPARD, J. (2002). Strategic Planning for Information Systems(3rd Edition) . Chichester: Wiley. [ Links ]

Received: April 04, 2018; Accepted: December 15, 2018

Address for correspondence: Maria Helena Monteiro, Centro de Administração e Políticas Públicas (CAPP), Instituto Superior de Ciências Sociais e Políticas (ISCSP), Universidade de Lisboa (ULisboa), Lisboa, Portugal. Email: helena.monteiro@iscsp.ulisboa.pt

Ricardo Ramos Pinto, Centro de Administração e Políticas Públicas (CAPP), Instituto Superior de Ciências Sociais e Políticas (ISCSP), Universidade de Lisboa (ULisboa), Lisboa, Portugal. Email: rrp@iscsp.ulisboa.pt

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License