Impact of training on the implementation of a new electronic system and acceptance of new technologies in a federal institution of higher education

Impacto do treinamento na implantação de novo sistema eletrônico e aceitação de novas tecnologias em uma instituição federal de ensino superior

ABSTRACT

Purpose:

The adoption of new technologies is a challenge for organizations, which leads them to invest in training when implementing a new system. The aim of this study was to evaluate the effect of the impact of training on the acceptance of a new system, as well as the relationship between the impact of training and the acceptance of new technology with the frequency of use of the system by the user in a federal institution of higher education.

Design/methodology/approach:

An online questionnaire was answered by participants from an institution two months after the training of a new system. A total of 288 responses were obtained (56.3 % were women, and 44.1 % were from 29 to 39 years). The scales for self-assessment of impact of training at work, for support to transfer, and for accepting new technologies were applied, in addition to sociodemographic data and frequency of use of the system. Correlations, linear regression, and ANOVA with a post hoc test were conducted to test the relationship between variables.

Findings:

The results indicated that the greater the perception of impact of training, higher the acceptance of the new system. In addition, those who demonstrated greater acceptance of technology, used the system more frequently.

Research limitations/implications:

This study has implications for the management of training related to new technologies.

Originality/value:

It also contributes to the field by bringing the concept of impact of training jointly with the acceptance of new technologies, which traditionally has investigated only reactions to training, and not the transfer of skills.

Keywords:
- technology acceptance; training; training impact; UTAUT; TAM

RESUMO

Objetivo:

A adoção de novas tecnologias é um desafio para as organizações, o que as leva a investir em treinamento quando implementam um novo sistema. O objetivo deste estudo foi avaliar o efeito do impacto do treinamento sobre a aceitação de um novo sistema, bem como a relação entre impacto do treinamento e de aceitação da nova tecnologia com a frequência de uso do sistema pelo usuário em uma instituição federal de ensino superior.

Desenho/metodologia/abordagem:

Um questionário online­ foi respondido pelos participantes da instituição dois meses após o treinamento sobre o novo sistema. Obtiveram-se 288 respostas (56,3 % mulheres e 44,1 % tinham entre 29 e 39 anos). Aplicaram-se as escalas de autoavaliação de impacto do treinamento no trabalho, escala de suporte à transferência de treinamento e a escala de aceitação de novas tecnologias, além de dados sociodemográficos e de frequência de uso do sistema. Para testar a relação entre as variáveis do modelo, foram verificadas correlações, regressão linear e ANOVA com teste post hoc.

Resultados:

Os resultados indicaram que quanto maior a percepção de impacto do treinamento, maior a aceitação do novo sistema. Ademais, aqueles que demonstraram maior aceitação da tecnologia utilizam mais frequentemente o sistema.

Limitações/implicações:

Este estudo gera implicações para a gestão de treinamentos relacionados a novas tecnologias.

Originalidade/valor:

Também contribui para o campo ao trazer o conceito de impacto do treinamento em conjunto com o de aceitação de novas tecnologias, que tradicionalmente investigou apenas reações ao treinamento, e não a transferência de habilidades.

Palavras-chave:
aceitação de tecnologia; treinamento; impacto do treinamento; UTAUT; TAM

1 INTRODUCTION

The Electronic Information System (Sistema Eletrônico de Informações - SEI), developed by the Federal Regional Court of the 4th Region (TRF4), is a platform that encompasses a set of modules and functionalities that promote administrative efficiency (Brasil, 2017). This system was adopted by the federal government as an electronic process solution under the National Electronic Process (NEP) project and features portability, remote access, external user access, access level control, administrative processing by multiple departments, specific features, and easy system navigability. As a regulation, Decree No. 8,539/2015 was published, which established the electronic processing of documents as a rule for direct, autonomous, and foundational public administration, with the SEI being given free of charge to related entities. Within the scope of the Federal Institutes of Higher Education (Instituições Federais de Ensino Superior - IFES), the Federal University of Rio Grande do Sul (UFRGS) and the University of Brasília (UnB) emerged as pioneers in the implementation of the SEI, whose implementation is still being studied by other IFES.

In order to comply with Decree No. 8,539/2015 and to promote speed and efficiency in their processes, the University of Brasília and the then Ministry of Planning, Budget and Management (MP) - which from 2019 became part of the Ministry of Economy - signed a technical cooperation agreement to implement the National Electronic Information System SEI and adhere to the National Electronic Process (NEP), which provided at the University for the replacement of physical processes by digital processes from May 16, 2016 (Agência Brasil, 2016). The implementation of a new system implied a change in organizational culture and demanded a commitment in the qualification of the workforce to handle that system.

In order to enable the workforce of all UnB units to be able to use the system before it starts operating, the IFES training area, in partnership with the system implementation commission, offered training in the use of the SEI. Bearing in mind that the implementation of a new system could affect the entire functioning of the processes in the organization, it was crucial that those who would operate the system were prepared to use it.

The training, development, and education (TD&E) literature indicates that training has as one of its objectives the preparation of workers to perform new functions and adapt to the introduction of new technologies (Meneses & Abbad, 2003). In this way, it can be understood that training has the potential to impact the acceptance of new technologies and digital systems. Furthermore, research on the unified theory of acceptance and use of technology, in particular the models proposed by Davis and Venkatesh, indicates that training can be a crucial factor in predicting the acceptance and intention to use new technologies (Harris, Mills, Fawson, & Johnson, 2016). However, there is a gap in studies that investigate the relationship between the impact of training in ampleness and the acceptance of new technologies. Previous research that evaluates this relationship has only used reactions to training as a measure (e.g. Harris et al., 2016), disregarding the assessment of transferring the skills acquired in training to the work context after training. Therefore, in order to fill this gap and expand the literature related to training and acceptance of new technologies, the objective of this research was to verify the effect of the training impact on the acceptance of a new electronic system (in this case, the SEI), as well as the relationship between the investigated variables and the frequency of use of the system by users in an IFES.

First, the perception of the impact of training at work is expected to affect the acceptance of the new system. For this, the following hypotheses were formulated for the study:

H1a: The greater the impact of training at work, the greater the perceived utility of the system.

H1b: The greater the impact of training at work, the greater the perceived ease of use of the system.

H1c: The greater the impact of training at work, the greater the relevance of the system to work will be perceived.

H1d: The greater the impact of training at work, the greater the perceived quality of the system’s results.

Regarding the frequency of use of the system, the literature points out that the acceptance of new technologies predicts the intention and behavior of using the technology (Williams, Rana, & Dwivedi, 2015). It is also expected that those who transfer the skills learned in training to work more, will use the system more frequently. Therefore, the following hypotheses are formulated:

H2a: The greater the perceived utility of the new system, the greater the frequency of use.

H2b: The greater the perceived ease of use of the new system, the greater the frequency of use.

H2c: The greater the relevance for the system’s work, the greater the frequency of use.

H2d: The greater the perceived quality of the system’s results, the greater the frequency of use of the new system.

H2e: The greater the impact of training at work, the greater the frequency of use of the system.

The justification for the interest in testing such hypotheses was due to the fact that there is a theoretical framework applicable to the study of the adoption and acceptance of technologies in organizations (Davis, 1989; Faria, Giuliani, Pizzinatto, & Pizzinatto, 2014; Friedrich, Basso, Froemming, & Baseggio, 2016; Venkatesh & Bala, 2008; Venkatesh & Davis, 2000; Venkatesh, Morris, Davis, & Davis, 2003; Venkatesh, Thong, & Xu, 2012), and it was realized that it would be pertinent to associate this literature to the evaluation of TD&E, considering that organizations spend resources and effort in systems training actions and it would be essential to assess whether the impact of training can influence the acceptance of a new technology.

The theoretical framework of the authors Davis and Venkatesh predicts that individuals will be able to manifest attitudes and behaviors of acceptance, intention to use, or even rejection of technologies from what they perceive as useful, easy to use, relevant to their work, and for the qualities of the results that the technological novelty may generate for their work, among other variables. Regarding the TD&E literature, it is understood that the impact of training is related to the concepts of transfer of training and performance at work (Freitas, Borges-Andrade, Abbad, & Pilati, 2006). The training impact can be defined as “the influence that the instructional event has on the training participant’s subsequent global performance, as well as on their attitudes and motivation” (Abbad & Pilati, 2005, p. 45). Thus, we can infer that given that the instructional event has an influence on the overall performance of the participant, it would also affect that employee’s acceptance of a new system, in this case, the SEI that would structurally modify how the processes are conducted within the institution.

Theories about acceptance of new technologies

The adoption of technologies aimed at computerization in Latin America, and in some cases, automation of processes and administrative routines, have gained increasing strength since the 1990s. From the first decades of the 21st century the public sector realized, with some delay, that it would be necessary to streamline processes, make them transparent and easily available to the population, eliminate paperwork, etc.

The literature (Davis, 1989; Venkatesh & Bala, 2008; Venkatesh & Davis, 2000; Venkatesh et al., 2003; Venkatesh et al., 2012), as mentioned previously, provides a very useful theoretical tool for the study of manifest acceptance by people at the time of adoption of new information and communication technologies (ICT) by organizations.

Based initially on the Theory of Reasoned Action (TRA) and the Theory of Planned Behavior proposed by Icek Ajzen since the 1970s, Davis (1989), Venkatesh and Davis (2000), Venkatesh et al. (2003), Venkatesh and Bala (2008), and Venkatesh et al. (2012) decided to formulate models, ranging from TAM (Technology Acceptance Model) to UTAUT (Unified Theory of Acceptance and Use of Technology) with a view to seeking to empirically understand what leads people to a positive attitude towards the adoption of new technologies, and that attitude would therefore lead them to the intention to use and the behavior of use in their full manifestation.

These authors and their theoretical propositions (TAM, UTAUT) suggest dozens of variables as predictors of the intention to use and the behavior of using technologies, such as: conditions that facilitate use, expectation of effort and performance regarding use, hedonic motivation, habit, price value, etc. (Venkatesh et al., 2012); as well as ease of use, perceived usefulness of technology, relevance of technology to the individual’s work, quality of the results that its adoption may generate, etc., as predicted by Venkatesh and Davis (2000), whose variables were considered for development of this study. Venkatesh and Bala (2008) propose that the relevance of a technology or system for work and the quality of results with the adoption of a technology are, in a way, determinants of the perceived usefulness by those adopting a technology. In this study, there is no such hierarchy, since the proposal is to relate the construct impact of training with the acceptance of new technologies, that is, to analyze whether when the employee actually improves their work after training, they will also evaluate the new implemented system in a more positive way.

In order to quickly conceptualize the study variables, we start with the concept of the variable called relevance to work, which corresponds to the degree to which an individual believes that the system is applicable in their work; quality of results is the degree to which an individual believes that the system allows better performance of their tasks; perceived ease of use corresponds to the degree to which an individual believes that the use of a technology will be effortless (Venkatesh & Bala, 2008, p. 277). The perceived utility refers to how much a person believes that using the system will increase their performance at work (Venkatesh & Davis, 2000). Finally, it is worth briefly considering the importance of including in Hypotheses 2a-2e of this work the variable frequency of use of the SEI as a dependent variable from the previous variables mentioned above. Venkatesh and other collaborators, in their studies of 2003 and 2012, do not neglect the importance of verifying the frequency of use of the technologies they choose to study. Venkatesh et al. (2012), for example, included in their research instrument the question: how often do you use these technologies (mobile e-mail, java games, browsing websites, SMS etc.)? And for this they proposed a frequency of use scale, which ranged from ‘never’ to ‘several times a day’.

Therefore, checking the daily frequency of use of the new system (SEI) and the daily frequency of use of other systems was crucial for this work. This is because knowing the frequency of use, it would be possible to estimate which user profile was predominantly found in the studied IFES, among the following profiles: Light-user, Medium-user and Heavy-user (Kotler, 2006, p. 253).

Friedrich et al. (2016) studied the relationship between support through training given by a bank to customers and the use of Internet banking, in a quantitative descriptive study aiming of 228 individuals over the age of 18, aiming to identify the impact of Internet banking training on the intention to use this system mediated by perceived risk, trust, and perceived ease of use. This study concluded that it is possible to infer that training in Internet banking indirectly expands the intention to use, by reducing the perceived risk and increasing the perceived ease of use and confidence.

The Current Population Survey (CPS), conducted in 2011 in the USA by the Census Bureau, which included questions about computer ownership, Internet use inside and outside the home, and the additional devices people use to go online, demonstrated that the level of education of individuals is related to the frequency of use of computers and the Internet. This reinforces the hypothesis that a subject’s training or proficiency may be a fact that leads them to use more information and technological resources today. With percentages not mutually exclusive, the study carried out with people aged 25 or over (this is only one of the age groups studied in the survey mentioned), found that 50.9 % of those who had at least one computer at home had less than high school as schooling, 70.9 % had graduated from high school, 84.3 % had undertaken some training in “college” and 93.1 % had bachelor’s degrees or had a college degree. With regard to accessing the Internet from outside the home, 31.5 % had less than high school education, 58.7 % had graduated from high school, 80.7 % had undertaken some college training and 90 % were bachelors or had a college degree. Note that as the educational level increases, so does the number of people using ICTs (e.g. computers and the Internet) (United States Census Bureau, 2013). In 2019, through the Internet usage in the United States report, it was found that the proportion of adults in the United States who are online ‘almost constantly’ (February 2019 data) increased with their higher graduation levels, namely: 23 % were in high school or less, 26 % were in some college and 36 % were college graduates or higher (Statista, 2019).

Ratchford & Ratchford (2014) when applying the TAP (Technology Adoption Propensity) index, a psychometric scale that measures four dimensions of consumers’ technological predispositions, examined how attitudes and dispositions influence the adoption patterns of technologies with 619 subjects, reinforce that:

Since it measures consumers’ perceived ability to successfully deal with technology, the proficiency subscale is a measure of experience in dealing with technology. Therefore, it is not surprising that it is related to the use of technology. It is known that experts in a domain form more refined knowledge structures on the topic (Mitchell & Dacin, 1996), search more efficiently (Moorthy, Ratchford, & Talukdar 1997), get more pleasure from the activities in which they have more experience (Celsi, Rose, & Leigh, 1993) and spend more time in activities in which they have more experience (Luo, Ratchford, & Yang, 2013) [...] This opens an avenue for future research (2014, p. 26).

This empirical evidence strengthens the hypothesis that subjects with a higher level of education, and therefore, more impacted by training efforts, studies etc., tend to use more technological resources at their disposal. This reflection supports hypotheses H2b and H2e in this study. In a way, all the hypotheses of the H2 group can find theoretical support for their propositions in Luo et al. (2013) who found in their study on the use of time of 287 consumers that they spend more time in activities in which they have greater expertise. For these authors, “experience also directly influences the value obtained from an activity” and the subject’s expertise generates a “chain of experience” and “value of the use of their time” (p. 24).

Impact of training at work

The TD&E area has been increasingly demanded and valued in federal educational institutions. In line with this growing demand, the enactment of decree No. 5,707/2006 establishing the Policy and Guidelines for Personnel Development of the direct, autonomous and foundational federal public administration helped to consolidate the area and brought a legal basis for the activities carried out by the training department of public agencies. This decree aims to “encourage and support the workforce in their training initiatives aimed at the development of institutional and individual competences”. In this sense, the high investment of financial resources to promote the training and qualification of the workforce points to the need for systematic evaluation of these actions in order to provide information, feedback, and improvement of the TD&E system (Borges-Andrade, Abbad, & Mourão, 2012).

In this context, training assessment would be the systematic process of collecting data that involves value judgment, providing the review, and improvement of institutional events through the analysis of existing instructional aspects (Goldstein, 1991, as mentioned in Meneses, Zerbini, & Abbad, 2010). Its main objectives are to gain control over the process, provide feedback to regardind the system, make decisions about training, and make it capable of causing changes in the training environment (Meneses, Zerbini, & Abbad, 2010).

Based on this concept, different training evaluation models have been proposed by the literature in the area. In the field of training assessment, it is important to highlight the Integrated Model for Assessing the Impact of Training at Work - IMPACT (Abbad, 1999). IMPACT comprises seven components: perception of organizational support; characteristics of training; characteristics of the clientele; reaction; learning; support to transfer competencies learned in training into work; and impact of training on the employees’ work. In this model, impact refers to the influence exerted by the training events on the participant’s subsequent performance in a work task similar to that learned in the training (Freitas et al., 2006). The impact can be in depth, when it includes the direct and specific effects of a TD&E action, or in ampleness, which assesses the indirect influence that this action has on the overall performance, attitudes and motivation - which is the construct used in this study (Zerbini et al., 2012). The instrument traditionally used in TD&E research to measure the impact in extent was developed by Abbad (1999) and later disclosed by Abbad, Borges-Andrade and Sallorenzo (2004). Subsequently, evidence of validity of the measure was demonstrated from a confirmatory factor analysis of the scale (Pilati & Abbad, 2005). The scale of impact of training in ampleness has a unifactorial structure evaluated in 12 items.

The antecedents of training impact have been extensively investigated in the literature, including aspects related to the environment, the individual, and the training itself. Much of the research evaluates variables related to the context, especially, environmental support. Those are the variables that have the greatest power to predict impact. Among them, the organizational environment, organizational support, learning support, and material and psychosocial support for transference stand out (Zerbini & Abbad, 2010).

Therefore, when addressing the impact of training, it is also important to consider the support for the transfer of the training that the employee will have, to apply the new skills acquired, which is one of the components of the IMPACT model. Support refers to the assistance the employee receives after training to transfer what they have learned to their work, that is, to apply the new skills (Abbad, Pilati, Borges-Andrade, & Sallorenzo, 2012). Traditionally, the support scale is composed of two factors: 1) Psychosocial support to transfer - refers to managerial, social, and organization support to the application of new skills; and 2) Material support for the transfer - refers to the assessment of the quality, quantity, and availability of material and financial resources. Brazilian research has shown a strong correlation between both psychosocial and material support with the impact of training (Abbad et al., 2012), reinforcing the idea that the transfer of skills learned in training depends largely on the social context in which the worker is and the resources available for them to apply the skills. Researchers found that the lack of transfer support affect learning scores and training impact at work (Carvalho & Abbad, 2006), another one indicated that support was highly related to training impact on the work of nurses (Bastos, Ciampone, & Mira, 2013). Due to the strong influence of support on impact of training, this variable was used as a control to evaluate the prediction model.

Having made the main theoretical considerations that support the study, the following section will present the methodological aspects of the study.

2 METHOD

This study can be classified as descriptive, through a case study with a quantitative approach with the workforce at a Brazilian IFES. The institution has about six thousand public employees on its staff, including professors and administrative technicians.

Participants

In this study, we used a convenience and non-probabilistic sample. There were 288 respondents, 162 of whom were women (56.3 %) and with a higher concentration in the age group from 29 to 39 years (44.1 %). Regarding education, most had specialization (lato sensu) (47.6 %) and belonged to the technical-administrative category (87.8 %). About 20% had less than one year of service and 26.7 % had between 1 to 3 years of service.

The participants were divided into three categories according to the frequency of use of the system as indicated by Kotler (2006). For category 1) Light-user, it was decided that this would be those users who used the system for up to 1 hour a day; 2) Medium-user, it was decided that this would be those users who used the system for a period of time between 1 and 4 hours a day; and 3) Heavy-user, it was decided that this would be those users who used the system for more than 4 hours a day. In the present survey, 96 participants were considered Light-user (33.3 %), 92 Medium-user (31.9 %), and 99 Heavy-user (34.4 %).

Instruments

The questionnaire applied was composed of the scales that evaluated the following constructs:

1. Impact of Training at Work Scale - Ampleness Measure (Abbad et al., 2012) composed of 12 items on a Likert scale of agreement from 1 to 5.

2. Support to Learning Transfer Scale (Abbad & Sallorenzo, 2001) with 12 items on managerial and social support for transfer of training and 5 items on material support for transfer on a Likert scale of frequency from 1 to 5.

3. Perceived Utility Scale with 4 items evaluated on a Likert scale of agreement from 1 to 7, translated and adapted by Venkatesh and Davis (2000).

4. Perceived Ease of Use Scale with 4 items assessed on a Likert scale of agreement from 1 to 7, translated and adapted by Venkatesh and Davis (2000).

5. Relevance to Work Scale with 3 items evaluated on a Likert scale of agreement from 1 to 7, translated and adapted by Venkatesh and Davis (2000).

6. Quality of Results Scale with 3 items evaluated on a Likert scale of agreement from 1 to 7, translated and adapted by Venkatesh and Davis (2000).

7. Demographic and functional data (gender, age group, education, years of service at the institution, functional category, daily frequency of use of the new system, and daily frequency of use of other systems).

We decided to use the original version of the scales related to the acceptance of the system by Venkatesh and Davis (2000), considering that the scales already translated and validated for Brazil did not specifically evaluate the use of the SEI system. Therefore, the instruments were translated, re-translated and adapted to the variable of interest for this research. The items of the instruments used in all scales are presented in Appendix A.

Procedures

The questionnaire was applied online by the IFES training platform. It was sent to the e-mails of all public employees who completed the course of the new system in 2016, which covers a population of 1,234 participants. In total, 288 public employees answered the questionnaire, which represents 23.3 % of the target audience.

The training course classes for the use of the new system were offered between March and November 2016 and each lasted 8 hours. The questionnaire was applied about two months after the offer of the last class of the course, being available for one month for completion by the participants of the training.

The survey was previously submitted for approval by IFES’s most senior people management authority. In the questionnaire instructions, the participants were informed that the responses were anonymous and confidential, treated in a grouped manner, without the identification of any respondent. In addition, it was clarified that participation in the research would be voluntary and that it would not have any consequences for their work.

Data analysis

Data were analyzed using SPSS Software version 26 to perform descriptive and inferential analyzes. In order to verify the psychometric quality of the impact and organizational support scales for this sample, confirmatory factor analyzes were conducted using the Mplus version 7.0 statistical package. Regarding the scale of interaction with the system, since it underwent modifications for this study, a parallel analysis and exploratory factorial analysis were first conducted to determine the factors, and confirmatory factor analysis of the instrument was then conducted.

Correlations, linear regression, and ANOVA were conducted to test the relationship between model variables. For the analysis of differences between groups in ANOVA, a post hoc test was performed. For this case, as there was no homogeneity of the variances, the Games-Howell test was used (Field, 2013).

Aiming at the quality of the analysis, the assumptions of normality were verified to proceed with linear regression (linearity, independence of errors, homoscedasticity of errors, and normal distribution of errors). Due to problems related to the normal distribution of errors, the training impact, perceived utility, perceived ease of use, and relevance to work variables were transformed using Box-Cox transformation (Osborne, 2010). Following the transformation, normality was achieved.

The data analyzes performed in this study are summarized in Table 1.

Table 1
Statistical procedures used in the study

Procedure	Data Analysis
Psychometric assessment of scales	Exploratory and confirmatory factor analysis
H1a-H1d hypotheses test	Simple linear regression
H2a-H2e hypotheses test	ANOVA with Games-Howell post-hoc test

3 RESULTS AND DISCUSSION

Psychometric Assessment of Scales

The scales of impact of training at work, perceived organizational support, and acceptance of new technologies were submitted to confirmatory factor analysis to confirm the theoretical model already presented in the literature of the area. The impact of training scale had as a theoretical model a single factor structure, while the organizational support scale consisted of two factors, one of psychosocial support and the other of material support (Abbad et al., 2012; Abbad & Sallorenzo, 2001).

In general, the model fitted the data very well. The impact of training at work scale remained with all items in a single factor structure, as indicated in the literature (Abbad et al., 2012). The fit indexes of the model were adequate [χ²(52) = 260; comparative fit index (CFI) = .97; Tucker-Lewis index (TLI) = .96; root mean square error of approximation (RMSEA) = .11], except for the value of RMSEA that was higher than that recommended in the literature which should be equal to or less than .06 (Thompson, 2004). However, it should be noted that previous research that assessed the scale’s structure (Pilati & Abbad, 2005) had already pointed out inadequate RMSEA indices.

The scale of organizational support was assessed in two factors: 1) Psychosocial support with 10 items; and 2) Material support with 5 items, as found in a previous study (Abbad & Sallorenzo, 2001). For this measure, the fit indexes of the model were satisfactory (χ²(89) = 339; CFI = .97; TLI = .96; RMSEA = .09), except for the RMSEA value that was slightly higher than recommended in the literature (Thompson, 2004).

For the new technology acceptance scale, four factors were extracted as indicated by the parallel analysis: 1) Perceived utility (α = .94) with 4 items; 2) Perceived ease of use (α = .87) with 4 items; 3) Relevance of the system (α = .88) with 3 items; and 4) Quality of the results generated by the system (α = .88) with 3 items. The scale factors obtained adequate reliability indexes. A confirmatory factor analysis of the scale was then conducted to confirm the structure of four factors already presented in the literature. The fit indices of the model were also satisfactory (χ²(70) = 234; CFI = .99; TLI = .99; RMSEA = .09), except for the RMSEA value that was slightly higher than recommended in the literature (RMSEA < .06) (Thompson, 2004).

The means and correlations between all variables investigated in this study are shown in Table 2.

Table 2
Descriptive data and correlation of the main study variables

	Mean	SD	1	2	3	4	5	6
1. Impact of training	3.72	0.78
2. Psychosocial support	3.51	0.81	0.49**
3. Material support	3.44	0.84	0.23**	0.36**
4. Perceived utility	5.84	1.42	0.63**	0.41**	0.20**
5. Perceived ease of use	5.48	1.26	0.59**	0.35**	0.12*	0.63**
6. Relevance to work	6.03	1.27	0.44**	0.30**	0.10	0.63**	0.51**
7. Quality of results	5.46	1.31	0.62**	0.42**	0.32**	0.68**	0.65**	0.53**

• Notes: SD = Standard Deviation. * p ≤ .05, ** p ≤ .01

Hypothesis testing

To verify the alternative hypotheses H1a-H1d, linear regression analyzes were conducted. The null hypothesis would be that there is no significant relationship between the impact of training and the construct of acceptance of new technology. Initially, the independent variable and the control variables (in this case, impact of training, psychosocial support, and material support) were centralized to avoid collinearity. Subsequently, in the first regression block, the variables of psychosocial and material support were inserted as control variables. In the second block, the variable of impact of training at work was inserted. The factors related to the acceptance of new technology were considered as a dependent variable. The results of the regressions to evaluate the H1a-H1d hypotheses are shown in Table 3.

Table 3
Linear hierarchical regression of the effect of the impact of training at work on the acceptance of new technologies

	Variable	Model 1 β	Model 2 β (IC 95%)
H1a DV: Utility	Psychosocial support	0.39**	0.13* (0.04-0.42)
	Material support	0.05	0.02 (-0.13-0.20)
	Training impact		0.56** (0.82-1.19)
	Adjusted R²	0.17	0.41
	ΔR²	0.18	0.24
	ΔF	30.35**	113.72**
H1b DV: Ease	Psychosocial support	0.36**	0.09 (-0.04-0.32)
	Material support	-0.01	-0.04 (-0.20-0.09)
	Training impact		0.56** (0.72-1.07)
	Adjusted R²	0.12	0.36
	ΔR²	0.13	0.24
	ΔF	20.53**	105.24**
H1c DV: Relevance	Psychosocial support	0.31**	0.12 (-0.01-0.39)
	Material support	-0.01	-0.03 (-0.21-0.13)
	Training impact		0.39** (0.43-0.82)
	Adjusted R²	0.09	0.20
	ΔR²	0.09	0.11
	ΔF	14.42**	40.11**
H1d DV: Quality	Psychosocial support	0.35**	0.09 (-0.02-0.33)
	Material support	0.19**	0.16* (0.09-0.40)
	Training impact		0.53** (0.72-1.06)
	Adjusted R²	0.20	0.41
	ΔR²	0.21	0.21
	ΔF	36.53**	103.22**

• Notes. DV = Dependent Variable; CI = Confidence Interval. * p < 0,05, ** p <0,001.

The results presented in Table 3 show evidence to support the H1a-H1d hypotheses. In all dimensions of acceptance of new technology, the impact of training at work had a positive and significant effect on the dependent variable variance. Furthermore, it can be observed that by including the impact of training variable, there was a significant improvement in the model’s prediction. The higher the adjusted R², the more the dependent variable (in this case, acceptance of the technology) is explained by the training. Therefore, the greater the perception of impact of training, the greater the acceptance of the new system implemented at the IFES. The result corroborates previous research that points to a positive relationship between training and acceptance of technologies. It also expands research to the concept of impact of training at work.

To verify the H2a-H2e hypotheses, an Analysis of Variance (ANOVA) was performed. The null hypothesis would be that there are no significant differences between groups regarding the impact of training and the acceptance of new technologies. The means compared between the groups of users are shown in Table 4. From the data analysis, it was found that there were significant differences between the groups regarding the perception of impact of training, and with respect to all dimensions of acceptance of new technologies. However, there were no significant differences between groups for the material support variable. Therefore, support was found for the H2a-H2e hypotheses, considering that the users who most accepted the new system and who perceived the greatest impact of the training, were those who used the SEI system the most.

In all variables analyzed, significant differences were found between Light-user and Medium-user as well as between Light-user and Heavy-user. However, no significant differences were found between Medium-user and Heavy-user. This result indicates that users who perceive greater impact of training as well as greater utility, ease of use, relevance, and quality of the system, use the new system more frequently.

As discussed in the theoretical section, these findings can be corroborated by the perspectives highlighted by the United States Census Bureau (2013), Ratchford & Ratchford (2014), and Luo et al. (2013), and especially regarding the ease of use and training impact (experience, expertise) as drivers for the greater time dedicated to the use of the technological tool.

Venkatesh and Bala (2008, p. 299) highlighted the importance of training for the perception of usefulness and ease of use of a technology by its user, when stating that: “training can be used to help users develop favorable perceptions regarding determinants of perceived utility and ease of use”. For the authors, training plays a critical role in the adoption and use of information technology, which is clear in the results obtained from the analyzes of intensity of use, presented in the profiles of users of the SEI.

Table 4
ANOVA for comparison of means between the types of users of the system

		N	Mean	SD	95 % Confidence Interval		F
					Lower limit	Upper limit
Training Impact	Light user	96	3.32	0.83	3.16	3.49	18.96**
	Medium user	92	3.85	0.66	3.71	3.99
	Heavy user	99	3.98	0.69	3.84	4.12
	Total	287	3.72	0.78	3.63	3.81
System Utility	Light user	96	4.88	1.53	4.57	5.19	33.20**
	Medium user	92	6.26	1.12	6.02	6.49
	Heavy user	99	6.35	1.06	6.14	6.57
	Total	287	5.83	1.42	5.67	6.00
Ease of Use	Light user	96	4.79	1.4	4.50	5.07	20.57**
	Medium user	92	5.76	0.95	5.56	5.96
	Heavy user	99	5.88	1.08	5.67	6.10
	Total	287	5.48	1.26	5.33	5.62
System relevance	Light user	96	5.11	1.52	4.80	5.42	34.52**
	Medium user	92	6.42	0.79	6.25	6.58
	Heavy user	99	6.55	0.82	6.38	6.71
	Total	287	6.03	1.27	5.88	6.17
System Quality	Light user	96	4.84	1.31	4.57	5.10	16.85**
	Medium user	92	5.72	1.17	5.48	5.97
	Heavy user	99	5.81	1.25	5.56	6.06
	Total	287	5.45	1.32	5.30	5.61

• ** p < 0,001

4 CONCLUSIONS

The results of this study demonstrated the importance of the new technology acceptance to intensify the use of the system. The literature in the area of ​​TD&E points out the need to evaluate the impact of training actions on the performance of people and organizations (Freitas et al., 2006) and that the trained employee should apply knowledge, skills, and attitudes properly at work. The research carried out corroborates the proposed hypotheses and indicates that the impact of training for a new system can be crucial for users to accept its implementation, and consequently, also use it more. This study expands the literature of the area by bringing the impact of training construct together with the literature of acceptance of new technologies, which traditionally has investigated only reactions to training, and not the transfer of skills.

Regarding the influence of the impact of training on the acceptance of technology, it is worth mentioning the study by Silva and Dias (2006) that examined the influence of user training on the acceptance of a business management system. In that study, training in the system had a direct effect on perceived ease of use, which directly affected perceived utility. Furthermore, the perceived utility was the main determinant of the positive attitude towards the system, which demonstrates that the perceived utility variable is one of the most important when evaluating the adoption of a new system.

As for the H2a-H2e hypotheses related to the influence of the new technology acceptance variables (perceived utility, perceived ease of use, relevance to work, and quality of the perceived results) and the impact of training on the frequency of use of the system were corroborated, although no differences were found between the user types Medium-user and Heavy-user.

It was observed that the more useful, easy, relevant, and higher quality of the system, the greater the frequency of use of this system. This indicates the importance of producing systems that are accessible to the employee. In addition, it was found that the training action was fundamental, since those who used the system more, evaluated a greater impact of the training, which means that that for them it was easier to transfer the acquired competences to the use of the system. However, it should be noted that the impact of training is only observed when the trained person applies what has been learned and when this application generates significant improvements at work (Freitas et al., 2006). In this study, it was observed that the trained employees are using the system more, however, it would be relevant in future studies to also evaluate the quality of the use of this system by the trained employees.

These findings corroborate previous studies, as discussed in the theoretical and discussion section of the results (Davis, 1989; Venkatesh & Bala, 2008; Venkatesh & Davis, 2000; Venkatesh et al., 2003, 2012). Among these, the study by Venkatesh and Bala (2008) stands out, for directly associating the acceptance of technologies and systems with the training of the adopter in the work environment. It is clear therefore that the theory focused on the adoption of technologies is still robust and pertinent to the study of the adoption of ICTs in complex organizations, and should be applied to other contexts and other technologies, as recommended by Venkatesh et al. (2012 ).

Some of the limitations of the study include the use of a quantitative case study approach that considers only one source and the bias of the monomethod, in addition to the use of self-report to ascertain the frequency of use of the system. It is also worth mentioning that the research was conducted in only one educational institution in a sample that was not representative of the total population. In this sense, it is suggested that further research should be conducted in other public organizations that are implementing the SEI to assess the perception of acceptance of the system. In addition, it would be important to include information from other sources in future research, such as the extraction of real data on the frequency of use of the system by users, and peer and leadership evaluation (hetero-evaluation of acceptance, intensity of use, and general evaluation of the relationship between training and acceptance and use of the system by the users involved).

Therefore, this study reinforces the importance of training combined with organizational support for the adoption and use of new technologies. It also points out that when implementing new systems, it is essential that they are easy to use and useful for the work of the employee who will use the system. When considering these aspects when designing a system, its adoption by the users will be facilitated, even more if it is combined with training that provides usage guidance. In addition, this study provides important support for managers within organizations, by reinforcing the importance of conducting training when implementing new technologies, in order to adequately prepare workers to handle new systems.

REFERÊNCIAS

• Abbad, G. S. (1999). Um modelo integrado de avaliação de impacto de treinamento no trabalho - IMPACT [An integrated model for assessing the impact of training at work - IMPACT] (Unpublished doctoral thesis). Universidade de Brasília, Brasília, Brasil.
• ABBAD, G; GAMA, A. L. G; & BORGES-ANDRADE, J. E. (2000). Treinamento: análise do relacionamento da avaliação nos níveis de reação, aprendizagem e impacto no trabalho [Training: analysis of the relationship of assessment in the levels of reaction, learning and impact at work]. Revista de Administração Contemporânea, 4(3), 25-45. doi:10.1590/S1415-65552000000300003
  » https://doi.org/10.1590/S1415-65552000000300003
• ABBAD, G; MENESES, P; & ZERBINI, T. (2010). Manual de treinamento organizacional [Organizational training manual]. Porto Alegre: Artmed.
• ABBAD, G; PILATI, R; BORGES-ANDRADE, J. E; & SALLORENZO, L. H. (2012). Impacto do treinamento no trabalho - medida em amplitude [Impact of on-the-job training - measured in breadth]. In Abbad, G. S; Mourão, L; Meneses, P. M. M; Zerbini, T; Borges-Andrade, J. E; Vilas-Boas, R. (Eds.), Medidas de avaliação em treinamento, desenvolvimento e educação: Ferramentas para gestão de pessoas (pp. 145-162). Porto Alegre: Artmed .
• ABBAD, G. D. S; & SALLORENZO, L. H. (2001). Desenvolvimento e validação de escalas de suporte à transferência de treinamento [Development and validation of training transfer support scales]. RAUSP Management Journal, 36(2), 33-45
• ABBAD, G. D. S; SALLORENZO, L. H; COELHO JUNIOR, F. A; ZERBINI, T; VASCONCELOS, L; & TODESCHINI, K. (2012). Suporte à transferência de treinamento e suporte à aprendizagem [Training transfer support and learning support]. In: Abbad, G. S; Mourão, L; Meneses, P. M. M; Zerbini, T; Borges-Andrade, J. E; Vilas-Boas, R. (Eds.), Medidas de avaliação em treinamento, desenvolvimento e educação: Ferramentas para gestão de pessoas (pp. 145-162). Porto Alegre: Artmed .
• AGÊNCIA BRASIL(2016, February). UnB implementa sistema que agiliza tramitação de documentos e processos [UnB implements a system that streamlines the processing of documents and processes]. Retrieved from https://agenciabrasil.ebc.com.br/geral/noticia/2016-02/unb-implementa-sistema-que-agiliza-tramitacao-de-documentos-e-processos
  » https://agenciabrasil.ebc.com.br/geral/noticia/2016-02/unb-implementa-sistema-que-agiliza-tramitacao-de-documentos-e-processos
• BASTOS, L. F. L; CIAMPONE, M. H. T; & MIRA, V. L. (2013). Avaliação de suporte à transferência e impacto de treinamento no trabalho dos enfermeiros [Assessment of transfer support and training impact on nurses’ work]. Revista Latino-Americana de Enfermagem, 21(6), 1274-81. doi:10.1590/0104-1169.2913.2364
  » https://doi.org/10.1590/0104-1169.2913.2364
• BORGES-ANDRADE, J. E; ABBAD, G. S; & MOURÃO, L. (2012). Modelos de avaliação e aplicação em TD&E [Evaluation and application models in TD&E]. In: Abbad, G. S; Mourão, L; Meneses, P. M. M; Zerbini, T; Borges-Andrade, J. E; Vilas-Boas, R. (Eds.), Medidas de avaliação em treinamento, desenvolvimento e educação: Ferramentas para gestão de pessoas (pp. 20-35). Porto Alegre: Artmed .
• BRASIL(2017). Ministério do Planejamento, Desenvolvimento e Gestão. Software Público Brasileiro. Retrieved from https://softwarepublico.gov.br/social/sei/sobre-o-sei
  » https://softwarepublico.gov.br/social/sei/sobre-o-sei
• BRASIL(2006). Decreto Nº 5.707, de 23 de fevereiro de 2006.
• CARVALHO, R. S; & ABBAD, G. (2006). Avaliação de treinamento a distância: reação, suporte à transferência e impactos no trabalho [Evaluation of distance training: reaction, transfer support and impacts at work]. Revista de Administração Contemporânea, 10(1), 95-116. doi:10.1590/S1415-65552006000100006
  » https://doi.org/10.1590/S1415-65552006000100006
• DAVIS, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 13(3), 319-340. doi:10.2307/249008
  » https://doi.org/10.2307/249008
• FARIA, L. H. L; GIULIANI, A. C; PIZZINATTO, N. K; & PIZZINATTO, A. K. (2014). Applicability of unified theory of acceptance and use of technology extended to the consumer use context (UTAUT2) in Brazil: An evaluation of the model using a sample of Internet users on smartphone. Revista de Administração da Universidade Federal de Santa Maria, 7(2), 332-348. doi:10.5902/1983465913088
  » https://doi.org/10.5902/1983465913088
• FIELD, A. (2013). Discovering statistics using IBM SPSS statistics. 4th Edition, London, Sage.
• FREITAS, I. D; BORGES-ANDRADE, J. E; ABBAD, G. D. S; & PILATI, R. (2006). Medidas de impacto de TD&E no trabalho e nas organizações. In Abbad, G. S; Mourão, L; Meneses, P. M. M; Zerbini, T; Borges-Andrade, J. E; Vilas-Boas, R. (Eds.), Medidas de avaliação em treinamento, desenvolvimento e educação: Ferramentas para gestão de pessoas (pp. 489-504). Porto Alegre: Artmed .
• FRIEDRICH, M. P. A; BASSO, K; FROEMMING, L. M. S; & BASEGGIO, H. W. (2016). O treinamento de clientes e sua relação com a intenção de uso do internet banking [Customer training and its relationship with the intention of using internet banking]. Revista Brasileira de Marketing, 15(5), 655-668. doi: 10.5585/remark.v15i5.3306
  » https://doi.org/10.5585/remark.v15i5.3306
• KOTLER, P; & KELLER, K. L. (2006). Administração de Marketing [Marketing Management] (12th edition). São Paulo: Pearson Prentice Hall.
• LUO, L; RATCHFORD, B. T; & YANG, B. (2013). Why we do what we do: A model of activity consumption. Journal of Marketing Research, 50(1), 24-43. doi:10.1509/jmr.10.0416
  » https://doi.org/10.1509/jmr.10.0416
• MENESES, P. P. M; & ABBAD, G. (2003). Preditores individuais e situacionais de auto e heteroavaliação de impacto do treinamento no trabalho [Individual and Situational Predictors of Self and Heteroevaluation of Training Impact on Job]. Revista de Administração Contemporânea, 7 (n.especial), 185-204. doi:10.1590/S1415-65552003000500010
  » https://doi.org/10.1590/S1415-65552003000500010
• PILATI, R; & ABBAD, G. (2005). Confirmatory factor analysis of a scale for impact of training at work. Psicologia: Teoria e pesquisa, 20(1), 31-38. doi:10.1590/S0102-37722005000100007
  » https://doi.org/10.1590/S0102-37722005000100007
• OSBORNE, J. W. (2010). Improving your data transformations: Applying the Box-Cox transformation. Practical Assessment, Research & Evaluation, 15(12), 1-9. doi: 10.7275/qbpc-gk17
  » https://doi.org/10.7275/qbpc-gk17
• RATCHFORD, M; & RATCHFORD, B. T. (2014). A Cross-Category Analysis of Dispositional Drivers of Technology Adoption. Vanderbilt Owen Graduate School of Management Research Paper, n. 2474908. doi:10.2139/ssrn.2474908
  » https://doi.org/10.2139/ssrn.2474908
• SILVA, A. L. M. R; & DIAS, D. S. (2006, September). Influência do treinamento de usuários na aceitação de sistemas ERP no Brasil [Influence of user training on the acceptance of ERP systems in Brazil]. Paper presented at the30ºEncontro da ANPAD - EnANPAD, Salvador, BA, Brasil. Paper retrieved from http://www.anpad.org.br/admin/pdf/enanpad2006-adic-0161.pdf
  » http://www.anpad.org.br/admin/pdf/enanpad2006-adic-0161.pdf
• STATISTA(2019, December). Internet usage in the United States. Retrieved from https://www.statista.com/statistics/497055/usa-adults-online-constantly-education/
  » https://www.statista.com/statistics/497055/usa-adults-online-constantly-education/
• THOMPSON, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington, DC: American Psychological Association.
• UNITED STATES CENSUS BUREAU(2013). Current Population Survey (CPS), Computer and Internet Use in the United States. Retrieved from https://www.infrastructureusa.org/wp-content/uploads/2013/06/p20-569.pdf
  » https://www.infrastructureusa.org/wp-content/uploads/2013/06/p20-569.pdf
• VENKATESH, V; & DAVIS, F. D. (1996). A model of the antecedents of perceived ease of use: Development and test. Decision sciences, 27(3), 451-481. doi:10.1111/j.1540-5915.1996.tb00860.x
  » https://doi.org/10.1111/j.1540-5915.1996.tb00860.x
• VENKATESH, V; MORRIS, M. G; DAVIS, G. B; & DAVIS, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478. doi:10.2307/30036540
  » https://doi.org/10.2307/30036540
• VENKATESH, V; & DAVIS, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46(2), 186-204. doi: 10.1287/mnsc.46.2.186.11926
  » https://doi.org/10.1287/mnsc.46.2.186.11926
• VENKATESH, V., & MORRIS, M. G. (2000). Why don’t men ever stop to ask for directions? Gender, social influence, and their role in technology acceptance and usage behavior. MIS Quarterly, 24(1), 115-139. doi:10.2307/3250981
  » https://doi.org/10.2307/3250981
• VENKATESH, V., & BALA, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision Sciences, 39(2), 273-315. doi:10.1111/j.1540-5915.2008.00192.x
  » https://doi.org/10.1111/j.1540-5915.2008.00192.x
• VENKATESH, V., THONG, J. Y., & XU, X. (2012). Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology. MIS Quarterly, 36(1), 157-178. doi:10.2307/41410412
  » https://doi.org/10.2307/41410412
• WILLIAMS, M. D., RANA, N. P., & DWIVEDI, Y. K. (2015). The unified theory of acceptance and use of technology (UTAUT): A literature review. Journal of Enterprise Information Management, 28(3), 443-488. doi: 10.1108/JEIM-09-2014-0088
  » https://doi.org/10.1108/JEIM-09-2014-0088
• ZERBINI, T., & ABBAD, G. (2010). Training transfer and work training impact: critical literature analysis. Revista Psicologia: Organizações e Trabalho, 10(2), 97-111.
• ZERBINI, T., COELHO JUNIOR, F. A., ABBAD, G. D. S., MOURÃO, L., ALVIM, S., & LOIOLA, E. (2012). Transferência de treinamento e impacto do treinamento em profundidade [Training transfer and impact of in-depth training]. Medidas de avaliação em treinamento, desenvolvimento e educação: Ferramentas para gestão de pessoas, p. 127-144, Porto Alegre: Artmed .

APPENDIX A

Table A1
Impact of Training in Ampleness Scale of Abbad et al. (2012)

Items
1. I often use what I was taught in training in my work.
2. I take the opportunities that I have to put into practice what I was taught in training.
3. The skills I learned in training made me make fewer mistakes in my work, in activities related to the training content.
4. I remember well the content taught in the training.
5. When I apply what I learned in training, I do my job faster
6. The quality of my work has improved in activities directly related to the training content
7. The quality of my work has improved even in those activities that did not seem to be related to the training content.
8. My participation in the training served to increase my motivation for work.
9. My participation in this training increased my self-confidence. (I now have more confidence in my ability to do my job successfully).
10. After my participation in the training, I have more often suggested changes in work routines.
11. The training I did made me more receptive to changes at work.
12. The training I did benefited my co-workers, who learned some new skills from me.

Table A2
Support to Learning Transfer Scale from Abbad and Sallorenzo’s (2001)

Items
1. I have the opportunity to use what I learned on the course at work
2. My manager encourages me to apply what I learned in training at work
3. My manager removes obstacles and difficulties in the effective use of new skills.
4. My manager encourages application.
5. My manager plans with me to use the new skills.
6. I receive the information necessary for the effective use of new skills.
7. My suggestions are taken into account.
8. I receive support from experienced colleagues.
9. My attempts to use the new skills go unnoticed.
10. Negative aspects of the application of new skills are highlighted.
11. I receive praise for the correct use of new skills.
12. I receive guidelines for the effective application of new skills.
13. The material resources are in good condition for use.
14. The organization provides the necessary material resources
15. Work tools are compatible with the effective application of new skills.
16. The workplace is suitable for applying new skills.
17. The organization provides sufficient resources.

Table A3
SEI System Acceptance Scale adapted from Venkatesh and Davis (2000)

Items
1. Using the SEI improves my job performance.
2. Using the SEI in my work improves my productivity.
3. Using the SEI improves my effectiveness at work.
4. I think the SEI is useful in my work.
5. My interaction with the SEI is clear and understandable.
6. Interacting with the SEI does not require much of my mental effort.
7. I find the SEI easy to use.
8. I find it easy to use the SEI to do what I want it to do.
9. In my work, the use of the SEI is important.
10. In my work, the use of the SEI is relevant.
11. The use of the SEI is relevant to the various tasks related to my work.
12. The quality of the results I receive from the SEI is high.
13. I have no problem with the quality of the results obtained in the SEI.
14. I rate the SEI results as excellent.

Contribution of authors.

Publication Dates

• Publication in this collection
  14 June 2021
• Date of issue
  Oct-Dec 2020

History

• Received
  14 May 2018
• Accepted
  10 Feb 2020