# Abstract

Distance programs are frequently used by public healthcare providers at the Unified Healthcare System (SUS) in Brazil. This research consisted of evaluating learning predictors in a distance specialization program offered by a university to public healthcare providers. The main objective of this study was to analyze the influence of variables related to the characteristics of clients (personal data, professional data, well-being at work) and their reaction to the course (instructional procedures and tutor performance) over the learning process. Data collection was conducted through the Internet, with the participation of 125 students. The averages for the items of the Well-being Scale, Reaction to Tutor Performance Scale and Reaction to the Instructional Procedures Scale remained near the midpoint of the Scale. Concerning the learning results, students obtained high averages (grades) in the evaluative activities of the program. Well-being and reaction questionnaires were valid based on the analyzes performed. Multiple regression analyzes showed that participant reaction to instructional procedures and their practice area significantly predicted the learning process of the sample. This work added new knowledge to the distance training evaluation area in public healthcare.

Keywords:
Distance learning; learning; Public Healthcare providers

# Resumo

Palavras-chave:
Educação a distância; aprendizagem; profissional de Saúde Pública

# Resumen

Acciones de enseñanza virtual son utilizadas frecuentemente por profesionales del Sistema Único de Salud (SUS). Esta investigación consistió en la evaluación de predictores del aprendizaje en un curso en línea ofrecido a profesionales de la salud por una Universidad. El objetivo del trabajo fue analizar la influencia de variables relacionadas con las características de los usuarios y la satisfacción con el curso en el proceso de aprendizaje. La recogida de datos fue realizada a través de Internet. Las medias de los ítems de la Escala de Bienestar de la Escala de Reacción al Desempeño del Tutor y de la Escala de reacción a los Procedimientos Instruccional es están cerca del punto medio de la escala. Con respecto al aprendizaje, se identificaron promedios altos en las actividades de evaluación de los alumnos del curso. Los cuestionarios de bienestar y de reacción se mostraron válidos a partir de los análisis realizados. Los análisis de regresión múltiple realizados muestran que la satisfacción con los procedimientos de formación y la actuación profesional predicen significativamente el proceso de aprendizaje. Se verificó que este trabajo agregó conocimientos para área de evaluación de entrenamiento a distancia en salud pública.

Palabras clave:
Enseñanza virtual; aprendizaje; profesional de Salud Pública

There is significant increase in the investment in education actions by Brazilian and foreign organizations, given that these actions can lead to benefits for individuals, teams, organizations and societies. Furthermore, current social, political, economic and organizational changes have caused work to be more cognitive, complex, fluid, uncertain, interconnected and invisible, demanding ever more the participation of workers in training, development and education actions (TD&E; Bell, Tannenbaum, Ford, Noe, & Kraiger, 2017Bell, B. S., Tannenbaum, S. I., Ford, J. K., Noe, R. A., & Kraiger, K. (2017). 100 Years of Training and Development Research: What we know and where we should go. Journal of Applied Psychology, 102(3), 305-323. doi: 10.1037/apl0000142
https://doi.org/10.1037/apl0000142...
).

To provide for the increasing demand for educational events, the education institutions and work organizations endeavor to meet the demands for individual and organizational competence development by means of promoting TD&E actions which, in turn, can bring greater profitability to the companies, as well as improve the workers´ capacity for decision-making and to better handle situations which involve stress, tension, frustration and conflicts. As result, the forms of delivery and teaching methodologies of educational actions are ever more diversified. Thus, distance learning (DL), mediated by information and communication technologies (ICT), is currently prominent. Therefore, universities are increasingly offering programs in the distance modality (Falola, Osibanjo, & Ojo, 2014Falola, H. O., Osibanjo, A. O., & Ojo, S. I. (2014). Effectiveness of training and development on employees’ performance and organizations competitiveness in the Nigerian Banking industry. Bulletin of the Transilvania University of Brasov. Economic Sciences, 7(1), 161-170. Retrieved from http://webbut.unitbv.ro/BU2014/Series%20V/BULETIN%20V%20PDF/22_FALOLA%2029.06-MARE.pdf
http://webbut.unitbv.ro/BU2014/Series%20...
; Martins & Zerbini, 2016Martins, L. B., & Zerbini, T. (2016). Fatores influentes no desempenho acadêmico de universitários em ações educacionais a distância. Estudos de Psicologia (Natal), 21(3), 317-327. doi: 10.5935/1678-4669.20160030
https://doi.org/10.5935/1678-4669.201600...
).

The increase in the number of distance programs offered by the organizations and education institutions and the high number of professionals who seek this mode of educational actions is potentialized by many factors: the emergence of new formative needs, the high cost of traditional education, the limitations to students derived from the work schedules and time expended with family obligations, difficulty in dislocation and geographical limitations, hindering the dislocation of potential students located in more distant regions. Furthermore, the increasing importance of continued learning along one´s life and the possibility of greater control over the rhythm of individual learning in DL enables this education modality to be frequently used by public institutions and to ensure the continuous education of different categories of workers. Public healthcare providers are included in this context (Logan, Lundberg, Roth, & Walsh, 2017Logan, J. W., Lundberg, O. H., Roth, L., & Walsh, K. R. (2017). The effect of individual motivation and cognitive ability on student performance outcomes in a distante education environment. Journal of Learning in Higher Education, 13(1), 83-91. Retrieved from http://files.eric.ed.gov/fulltext/EJ1139727.pdf
http://files.eric.ed.gov/fulltext/EJ1139...
; Mira et al., 2011Mira, V. L., Peduzzi, M., Melleiro, M. M., Tronchin, D. M. R, Prado, M. F. F., Santos, P. T., Lara, E. M. S., ...Borges-Andrade, J. E. (2011). Análise do processo de avaliação da aprendizagem de ações educativas de profissionais de enfermagem [Special issue]. Revista da Escola de Enfermagem da USP, 45, 1574-1581. doi: 10.1590/S0080-62342011000700006
https://doi.org/10.1590/S0080-6234201100...
).

Institutional actions can expand the meaning of work in healthcare and promote the accountability of the professional of this area through the provision of quality service, insofar as it places the worker as subject of their own actions and facilitates their access to the labor market (Vieira & Chinelli, 2013Vieira, M., & Chinelli, F. (2013). Relação contemporânea entre trabalho, qualificação e reconhecimento: Repercussões sobre os trabalhadores técnicos do SUS. Ciência & Saúde Coletiva, 18(6), 1591-1600. doi: 10.1590/S1413-81232013000600011
https://doi.org/10.1590/S1413-8123201300...
). However, in Brazil, few studies are dedicated to investigating educational actions in healthcare. Thus, a large portion of institutional events in healthcare lack evaluation and, consequently, there is no monitoring of knowledge, abilities and attitudes (KAA) acquired by means of training (Quaglia, Oliveira, & Velho, 2015Quaglia, I., Oliveira, A., & Velho, A. P. M. (2015). Capacitação em saúde na Educação a Distância (EAD): Uma análise sistemática do conteúdo. Saúde e Pesquisa, 8, 103-112. Retrieved from http://periodicos.unicesumar.edu.br/index.php/saudpesq/article/view/3767/2516
http://periodicos.unicesumar.edu.br/inde...
).

The evaluation of training is a vital stage to ensure the improvement of institutional actions, and can guarantee more strategic TD&E actions, given that, from this subsystem, it is possible to identify if the programs are effective, which are the restricting or facilitating aspects of trainee performance, and how much the educational action ensures financial feedback to the organization, offering subsidies for the organizations to monitor its educational actions in a more effective form (Yanssen & Khanfar, 2009Yanssen, Z., & Khafar, M. (2009). Sales training effectiveness: Managers perceptions in the United Arab Emirates organizations. The Business Review, 13(2), 120-128. doi: 10.1108/MIP-02-2013-0029
https://doi.org/10.1108/MIP-02-2013-0029...
). Thus, TD&E measuring instruments are increasingly used to evaluate the characteristics of the clients, their reaction to the program and learning level (Mourão & Menezes, 2012Mourão, L., & Meneses, P. P. M. (2012). Construção de Medidas em TD&E. In G. S. Abbad, L. Mourão, P. P. M. Meneses, T. Zerbini, J. E. Borges-Andrade, & R. Vilas-Boas (Eds.), Medidas de Avaliação em Treinamento, Desenvolvimento e Educação (pp. 50-63). Porto Alegre: Artmed.).

Client characteristics represent a set of important trainee variables for the results, especially in the context of DL, which attracts a group of students with distinct profiles and personalities (Garavan, Carbery, O´Malley, & O´Donnell, 2010Garavan, T. N., Carbery, R., O’Malley, G., & O’Donnell, D. (2010). Understanding participation in e-learning in organizations: A largescale empirical study of employees. International Journal of Training and Development, 14(3), 155-168. doi: 10.1111/j.1468-2419.2010.00349
https://doi.org/10.1111/j.1468-2419.2010...
). Among these characteristics are the sociodemographic variables, which refer to the personal and professional profile of the participant, as well as psychosocial variables, which include self-referent variables such as well-being at work (Meneses, Zerbini, & Abbad, 2010Meneses, P. P. M., Zerbini, T., & Abbad, G. (2010). Manual de treinamento organizacional. Porto Alegre, RS: Artmed.).

Well-being at work consists of the prevalence of positive emotions at work and of the individual´s perception that, in one´s work, one expresses his/her potentials and progresses to reach their life goals (Paschoal & Tamayo, 2008Paschoal, T., & Tamayo, A. (2008). Construção e validação da Escala de bem-estar no trabalho. Avaliação Psicológica, 7(1), 11-22. Retrieved from http://www.redalyc.org/articulo.oa?id=335027183004
http://www.redalyc.org/articulo.oa?id=33...
). Well-being at work is a psychosocial variable of significant importance for evaluating educational actions, since the psychic pleasure perceived in work is an example of input, with one of the aspect derived from the positive affections perceived in the working environment (Borges-Andrade, 2006Borges-Andrade, J. E. (2006). Avaliação integrada e somativa em TD&E. In J. E. Borges-Andrade, G. S. Abbad, & L. Mourão (Eds.), Treinamento, desenvolvimento e educação em organizações e trabalho: Fundamentos para a gestão de pessoas (pp. 343-358). Porto Alegre, RS: Artmed.; Tims, Bakker, & Derks, 2013Tims, M., Bakker, A. B., & Derks, D. (2013). The impact of job crafting on job demands, job resources, and well-being. Journal of Occupational Health Psychology, 18(2), 230-240. doi: 10.1037/a0032141
https://doi.org/10.1037/a0032141...
). However, the well-being at work variable has not been investigated as a predictor of learning.

In addition to the client characteristics, participant reaction is another important level for evaluating educational actions, given that it reflects the satisfaction of the student with the rhythm, form, organization of the program and with how much was learned during training, and can be considered the perceptual results of a program (Bhatti, Ali, Isa, & Battour, 2014Bhatti, M. A., Ali, S., Isa, M. F. M., & Battour, M. M. (2014). Training Transfer and Transfer Motivation: The influence of individual, environmental, situational, training design, and affective reaction factors. Performance Improvement Quarterly, 27(1), 51-82. doi: 10.1002/piq.21165
https://doi.org/10.1002/piq.21165...
). According to Phillips and Phillips (2016)Phillips, J. J., & Phillips, P. P. (2016). Handbook of training evaluation and measurement methods (4th ed.). New York: Routledge., it is vital to evaluate the satisfaction of the workers with institutional actions, since it will influence the satisfaction with the work of the trainees and the decision-making process in their professional career, such as the choice for remaining in employment or not. The reaction to training programs can, thus, be a determining factor for retaining talents in the organization. Among the aspects evaluated by the reaction in the e-learning context are: reaction to institutional procedures, which refer to the level of satisfaction of the participant regarding the characteristics of the institutional event, and reaction to tutor performance, which measures the opinions of the trainees on tutor performance (Martins & Zerbili, 2015Martins, L. B., & Zerbini, T. (2015). Evidências de validade de instrumentos de reações no ensino superior à distância. Psicologia do Desenvolvimento, 15(1), 116-134. doi: 10.5935/1678-4669.20160030
https://doi.org/10.5935/1678-4669.201600...
).

Many studies show that the variables regarding client characteristics and their reactions can be learning predictor variables (Gotardo, Souza, Hruschka, & Viana, 2012Gotardo, R. A., Souza, H. A., Hruschka, E., Jr., & Viana, D. B. G. (2012). Teorias de aprendizagens na EAD: Fundamentação no uso dos recursos de design instrucional e design interacional. Paper presented at the SIED Simpósio Internacional de Educação a Distância (pp. 1-13). São Carlos, SP. Retrieved from http://sistemas3.sead.ufscar.br/ojs/index.php/sied/article/view/365
; Persky et al., 2009Persky, S., Kaphingst, K. A., McCall, C., Lachance, C., Beall, A. C., & Blascovich, J. (2009). Presence relates to distinct outcomes in two virtual environments employing different learning modalities. Cyberpsychology & Behavior, 12(3), 263-268. doi: 10.1089/cpb.2008.0262
https://doi.org/10.1089/cpb.2008.0262...
; Prichard, Bizo, & Stratford, 2011Prichard, J. S., Bizo, L. A., & Stratford, R. J. (2011). Evaluating the effects of team-skills training on subjective workload. Learning and Instruction, 21(3), 429-440. doi: 10.1016/j.learninstruc.2010.06.003
https://doi.org/10.1016/j.learninstruc.2...
). Thus, learning is also essential for evaluating DL actions, given that this variable becomes increasingly indispensable for the development of an individual career and for the success of the organizations (Manuti, Pastore, Scardigno, Giancaspro, & Morciano, 2015Manuti, A., Pastore, S., Scardigno, A. F., Giancaspro, M. L., & Morciano, D. (2015). Formal and informal learning in the workplace: A research review. International Journal of Training and Development, 19(1), 1-17.doi: 10.1111/ijtd.12044
https://doi.org/10.1111/ijtd.12044...
). Learning is consolidated when behavioral changes are verified and remain. These changes can occur in the cognitive, psychomotor and attitudinal domains in answer to the need for work and to develop competences relevant for future activities. However, learning from TD&E processes is still little understood (Brandão, Borges-Andrade, & Guimarães, 2012Brandão, H. P., Borges-Andrade, J. E., & Guimarães, T. A. (2012). Desempenho organizacional e suas relações com competências gerenciais, suporte organizacional e treinamento. Revista de Administração, 47(4), 523-539. doi: 10.5700/rausp1056
https://doi.org/10.5700/rausp1056...
).

Thus, the present article has the general objective of analyzing the influence of variables related to client characteristics (personal data, professional data and well-being at work) and their reaction to the program (reaction to institutional procedures an reaction to tutor performance) in the learning process of professionals from public healthcare. In this study, two research hypotheses were considered. The first hypothesis defends that the professionals with higher levels of well-being at work and more satisfied with the institutional procedures and with tutor performance would demonstrate greater learning. The second hypothesis defends that learning can be predicted by age, time of service, area of performance and professional formation, salary range, the fact of having conducted another graduation program and the fact he/she performs or intends to perform in public healthcare. Furthermore, this article had the specific objective of verifying evidences of the validity of measurement instruments using the Well-being at work Scale described by Paschoal and Tamayo (2008)Paschoal, T., & Tamayo, A. (2008). Construção e validação da Escala de bem-estar no trabalho. Avaliação Psicológica, 7(1), 11-22. Retrieved from http://www.redalyc.org/articulo.oa?id=335027183004
http://www.redalyc.org/articulo.oa?id=33...
, Reaction of tutor performance Scale (Martins, 2016Martins, L. B. (2016). Impacto do treinamento no trabalho: Determinantes individuais e contextuais em cursos corporativos a distância (Doctoral dissertation, Faculdade de Filosofia, Ciências e Letras de Ribeirão Preto, Universidade de São Paulo, Ribeirão Preto, SP, Brazil). Retrieved from http://www.teses.usp.br/teses/disponiveis/59/59137/tde-07112016-110039/pt-br.php
http://www.teses.usp.br/teses/disponivei...
) and Reaction to institutional procedures Scale (Martins, 2016Martins, L. B., & Zerbini, T. (2016). Fatores influentes no desempenho acadêmico de universitários em ações educacionais a distância. Estudos de Psicologia (Natal), 21(3), 317-327. doi: 10.5935/1678-4669.20160030
https://doi.org/10.5935/1678-4669.201600...
).

The research consisted in the evaluation of learning in a distance specialization program offered to public healthcare providers by a public university in Brazil. The objective of this educational action was to pedagogically form healthcare providers to perform as teachers in institutional events of mid-level technical professional education in the healthcare area, as well as qualify them for an effective and permanent education in healthcare (PEH). The program presented study-load of 660 hours. The curriculum was comprised of a nuclei which included 10 modules, each characterized by one theme.

The evaluated program is situated in a context of PEH, since it encompasses transformative and problematic healthcare practices, and covers the reality of the services, aiming at promoting integration between teaching and work in public healthcare (Miccas & Batista, 2014Miccas, F. L., & Batista, S. H. S. S. (2014). Educação permanente em saúde: Metassíntese. Revista de Saúde Pública, 48(1), 170-185. doi:10.1590/S0034-8910.2014048004498
https://doi.org/10.1590/S0034-8910.20140...
). It consisted of an institutional action executed after formation in undergraduate level, with the objective of providing updates of KAA to the professionals. Furthermore, the target-program is not corporative and occurs in a context of professional qualification, as an action for inducing learning.

# Method

## Participants

The research sample was comprised of students of the target-program, with no distinction of gender, time of service or other characteristics. Participation was voluntary, characterizing the sample as non-random and of convenience, since it included every individual that excepted participation, among those who fulfilled the inclusion criteria. All participants of the educational action were contacted in order to obtain their full participation. The target-public of the program was comprised of professionals graduated in healthcare (nurses, physiotherapists, doctors, nutritionists, psychologists, among others). The total of participants in this educational action was of 225 students. Among all academics, 125 answered the questionnaires. The feedback index was of 56%.

## Instruments

With the objective of surveying the client characteristics data, questionnaires that investigate personal and professional data of the students and the well-being at work scale developed by Paschoal and Tamayo (2008)Paschoal, T., & Tamayo, A. (2008). Construção e validação da Escala de bem-estar no trabalho. Avaliação Psicológica, 7(1), 11-22. Retrieved from http://www.redalyc.org/articulo.oa?id=335027183004
http://www.redalyc.org/articulo.oa?id=33...
were used. For collecting the data of participant reaction, the instruments included reaction to institutional procedures scale (Martins, 2016Martins, L. B., & Zerbini, T. (2016). Fatores influentes no desempenho acadêmico de universitários em ações educacionais a distância. Estudos de Psicologia (Natal), 21(3), 317-327. doi: 10.5935/1678-4669.20160030
https://doi.org/10.5935/1678-4669.201600...
) and the reaction to tutor performance scale (Martins, 2016Martins, L. B., & Zerbini, T. (2016). Fatores influentes no desempenho acadêmico de universitários em ações educacionais a distância. Estudos de Psicologia (Natal), 21(3), 317-327. doi: 10.5935/1678-4669.20160030
https://doi.org/10.5935/1678-4669.201600...
).

The well-being at work scale is comprised of two parts: one of affection and the other of expressivity/fulfillment at work. The positive and negative affection factors are evaluated based on the Likert scale which varies between 1 (not even a little) and 5 (extremely). The fulfillment/expressivity factor is also evaluated by means of the Linkert scale, ranging from 1 (completely disagree) to 5 (completely agree). The instrument was comprised by three factors: factor 1 - positive affection: with nine items and Cronbach´s alpha reliability coefficient of 0.93; factor 2 - negative affection: with twelve items and Cronbach´s alpha reliability coefficient of 0.91; and factor 3 - fulfillment/expressivity: with nine items and Cronbach´s alpha reliability coefficient of 0.88 (Paschoal & Tamayo, 2008Paschoal, T., & Tamayo, A. (2008). Construção e validação da Escala de bem-estar no trabalho. Avaliação Psicológica, 7(1), 11-22. Retrieved from http://www.redalyc.org/articulo.oa?id=335027183004
http://www.redalyc.org/articulo.oa?id=33...
).

To comprise the reaction evaluation measure, reaction instruments constructed by Martins (2016)Martins, L. B., & Zerbini, T. (2016). Fatores influentes no desempenho acadêmico de universitários em ações educacionais a distância. Estudos de Psicologia (Natal), 21(3), 317-327. doi: 10.5935/1678-4669.20160030
https://doi.org/10.5935/1678-4669.201600...
were used, based on adaptation and revalidation of the reaction to tutor performance in DL scale described by Zerbini and Abbad (2009a)Zerbini, T., & Abbad, G. S. (2009a). Reação ao desempenho do tutor em um curso a distância: Validação de uma Escala. Estudos e Pesquisas em Psicologia, 9(2), 447-463. Retrieved from http://www.e-publicacoes.uerj.br/index.php/revispsi/article/view/9114/7496
http://www.e-publicacoes.uerj.br/index.p...
and reaction to institutional procedures in DL scale describes by Zerbini and Abbad (2009b)Zerbini, T., & Abbad, G. S. (2009b). Reação aos procedimentos instrucionais de um curso via internet: Validação de uma escala. Estudos de Psicologia (Campinas), 26(3), 363-371. doi: 10.1590/S0103-166X2009000300009
https://doi.org/10.1590/S0103-166X200900...
, done by Martins (2012)Martins, L. B. (2012). Aprendizagem em ações educacionais a distância: Fatores influentes no desempenho acadêmico de universitários (Master thesis, Faculdade de Filosofia, Ciências e Letras de Ribeirão Preto, Universidade de São Paulo, Ribeirão Preto, SP, Brazil). Retrieved from http://www.teses.usp.br/teses/disponiveis/59/59137/tde-02102012-123253/pt-br.php
http://www.teses.usp.br/teses/disponivei...
.The new adaptation of the reaction to tutor performance scale now contains six items, Cronbach α of 0.78 and factorial loads between 0.54 and 0.78. This instrument was applied only to a pilot study with undergraduates and graduates in Spain. The reaction to institutional procedures scale, after adaptation, presented 12 items. In this work, the author did not present the instrument´s reliability analysis study (Martins, 2016Martins, L. B., & Zerbini, T. (2016). Fatores influentes no desempenho acadêmico de universitários em ações educacionais a distância. Estudos de Psicologia (Natal), 21(3), 317-327. doi: 10.5935/1678-4669.20160030
https://doi.org/10.5935/1678-4669.201600...
).

For both reaction instruments, the items were answered according to Likert scale of 5 points, ranging from 1 (very bad) to 5 (very good). The reaction questionnaires presented reduced number of items, therefore, are more economic, faster, easy to apply and consonant to those existing in international literature (Martins, 2016Martins, L. B., & Zerbini, T. (2016). Fatores influentes no desempenho acadêmico de universitários em ações educacionais a distância. Estudos de Psicologia (Natal), 21(3), 317-327. doi: 10.5935/1678-4669.20160030
https://doi.org/10.5935/1678-4669.201600...
). Learning was evaluated based on the grades obtained by the students in evaluative activities, by means of calculating the mean of the sum of the grades obtained by the students in each of the 10 modules of the specialization program.

## Ethic Procedures

The research project was submitted to platform Brasil and approved under Ethics Appreciation Presentation Certificate (CAAE) nº 41301115.0.0000.5407, also approved by the Ethics in Research Committee (ERC). The participants were ensured of the right to anonymity, of the voluntary character of participation in the study and that the data collected would be used exclusively for scientific publication.

## Data Collection Procedure

Data collection was done remotely, by internet, from July to August of 2015, using the link made available in the program´s virtual learning environment platform or by means of invites for participating in the research sent by personal e-mail. At the end of the data collection, the program coordination was asked to send worksheets with the grades of the students in order to measure their learning.

## Data Analysis Procedure

The measurement instruments used in this research, as well as the grades the students obtained in the evaluative activities, were part of the study database. The statistical analyses of the quantitative data were performed using the Statistical Package for Social Sciences (SPSS) software, version 22, and included the following procedures: descriptive and exploratory data analysis, exploratory factorial analysis (EFA) and standard multiple regression analysis.

The descriptive and exploratory data analyses were performed by means of data entry analysis, of the presence of extreme cases, of the distribution of omissive cases, of the frequency distribution of the variables and of the size of the sample. The standard multiple regression analyses were conducted to test the set of relations between the constructs of the proposed investigation model, aiming at analyzing the influence of the variables related to the client characteristics and the reaction to the program regarding learning.

The EFA were conducted using the Principal Components (PC), Principal Axis Factoring (PAF) and internal consistence (Cronbach alpha) methods to verify the evidences of validity and reliability of the measurement instruments. The EFA of the empirical structures of the questionnaires involved the identification of omissive and extreme, univariate and multivariate cases, linearity, singularity and multicollinearity analyses, the size of the correlations and the Kaiser-Meyer-Olkin (KMO) test, the initial factor extraction by PC analysis, structural factorial analysis by PAS and the calculation of the Cronbach alpha and factorial scores. Therefore, many procedures were performed to explain the correlation or covariance between a set of variables.

# Results

Concerning the profile of the participant students, most of the sample is constituted by women (88.9%), married individuals (46.0%), presenting average age of 33.8 years (SD=7.6). Regarding the time of service, the predominance was of participants with 1 to 5 years (49.2%). Most perform or intend to perform in public healthcare (86.5%). They currently present employment relation (86.5%) and perform professionally, overall in nursing (47.2%). In addition to coursing the specialization, most have already undergone a graduation program (70.6%).

In relation to well-being at work, among the 12 items of negative affection, there was higher concentration of answers in point 2 of the scale. For the items of positive affection, a higher con-centration of answers occurred in point 3 of the scale. Of the nine items present in the fulfillment/expressivity factor, seven answered point 4 in the scale. There was higher concentration of answers for reaction to tutor performance and reaction to institutional procedures in point 4 of the scale. It was possible to verify that the means of the items of the well-being at work scale, reaction to tutor performance scale and reaction of institutional procedures scale were located near the mid-point of the scale.

Regarding the evaluation of student learning, the grades of 136 healthcare providers who obtained specialist title were analyzed. It is noteworthy that the evaluation of each module was graded between 0 and 100 points. The means of the grades per module ranged from 83.6 to 88.5. High grade averages occurred in all modules. Although the present work did not propose to analyze the evaluative activities of the programs, it is necessary to critically reflect on these results. In this study, it was impossible to affirm if the grades obtained by the students truly reflect the KAA learning acquired during the educational action.

From the EFA of the well-being at work scale described by Paschoal and Tamayo (2008)Paschoal, T., & Tamayo, A. (2008). Construção e validação da Escala de bem-estar no trabalho. Avaliação Psicológica, 7(1), 11-22. Retrieved from http://www.redalyc.org/articulo.oa?id=335027183004
http://www.redalyc.org/articulo.oa?id=33...
, it was unnecessary to modify the questionnaire or exclude any items, given that all items of the scale significantly represented the factor. Factor 1, positive affection, presents nine items and Cronbach α of 0.95. Factor 2, negative affection, presents 12 items and Cronbach α of 0.93. Factor 3 presented nine items and internal consistency index of 0.93. By means of the EFA of the evaluation instrument reaction to tutor performance, the scale remained with six items and was comprised of 1 factor with Cronbach α of 0.09. The items presented factorial loads ranging from 0.63 to 0.82. After the EFA, the questionnaire for evaluating reaction to institutional procedures maintained the 12 items, with unifactorial structure, factorial loads ranging from 0.52 to 0.85. All items of the scale significantly represented the factor (Cronbach α of 0.91).

Standard Multiple Regression Analyses were conducted to reach the general objective of the study, which was to analyze the influence of variables related to the client characteristics and reaction to the program over the learning process. According to criteria described by Tabachnick and Fidell (2007)Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. New York: Harper Collins College., the regression analyses can be performed only when the sample is superior or equal to 50 cases, plus eight times the number of antecedent variables (N≥50 + 8m) to test the multiple correlations. In the investigated target-program, the number of participants was of 116. Thus, it was possible to perform regression analysis with 5 antecedent variables in Model A and with 8 antecedent variables in Model B.

The complete data file of the participants was used for the regression analyses after having been submitted to initial exploratory analyses and the factorial scores having been produced from the mean of the original variables of the factors by means of EFA, containing the multivariate model with the paired antecedent variables. For Model A, the following antecedent variables were included in the analyses: positive affection, negative affection, fulfillment/expressivity, reaction to tutor performance, reaction to institutional procedures (N=116; 5 variables) and variable learning criteria (Figure 1).

Figure 1
Model A of Learning prediction analysis.

Table 1 presents the standard multiple regression for investigation model A. The multiple regression coefficient (R) was significantly different from zero, with confidence limits of 95%. The antecedent variable which significantly contributes for the learning explanation, considering interval of confidence of 95%, was reaction to institutional procedures (β=0.29), in order that the learning of the participants of the target-program is explained by the reaction to institutional procedures (29.0%).

Table 1
Standard Multiple Regression for Investigation Model A

Model B, which can be visualized in Figure 2, had included into the analyses the following antecedent variables, which comprise the client characteristics: age, area of professional performance, time of service, salary range, undergraduate program, other graduation program coursed, performance of intent to perform in public healthcare, and employment relation (N=116; 8 variables) and the learning criteria variable.

Figure 2
Model B for Learning prediction analysis.

Table 2 demonstrates the standard multiple regression for investigation model B. The multiple regression coefficient (R) was significantly different from zero, with confidence limits of 95%. The antecedent variable which significantly contributed for explaining the learning levels, considering the interval of confidence of 95%, was area of professional performance (β=-0.26), in order that the learning levels of the participants of the target-program is explained by the professional performance of healthcare providers (26.0%).

Table 2
Standard Multiple Regression for Investigation Model B

In sum, based on the regression analysis, it was verified that the participant who presented positive reactions to the institutional procedures and who performs in certain professional area obtained the highest grades in the evaluative activities and, as a result, presented higher levels of learning. However, for the present sample, the positive affection, negative affection, fulfillment/expressivity, reaction to tutor performance, age, time of service salary range, undergraduate program, other graduate program coursed, performance or intent to perform in public healthcare and employment relation variables did not significantly influence learning.

# Discussion

The EFA of the empirical structures of the questionnaires were very useful, given that, from the analyses of the descriptive data of reaction to tutor performance, reaction to institutional procedures and well-being at work measures, a tendency of the participants to answer the items with values close to the mid-point of the scale occurred. Thus, it was important to verify the evidences of validity of the scales to confirm if the research presented central tendency error in order to demonstrate if the questionnaires measure what they supposedly should.

The well-being at work instrument was shown to be reliable and valid for evaluating the client characteristics of the trainees, and can easily be applied for diagnosing the effectivity of the distance educational actions in the context of public healthcare. Based on the EFA, is was possible to confirm the definition given by Paschoal and Tamayo (2008)Paschoal, T., & Tamayo, A. (2008). Construção e validação da Escala de bem-estar no trabalho. Avaliação Psicológica, 7(1), 11-22. Retrieved from http://www.redalyc.org/articulo.oa?id=335027183004
http://www.redalyc.org/articulo.oa?id=33...
which considers well-being at work as a result of affection aspects (positive and negative affections) and cognitive aspects (expressivity and fulfillment perception), thus confirming a tendency verified in international literature in prioritizing the affection dimension and the personal expression dimension, as well as work performance, to the concept of well-being. Therefore, it was verified that well-being involves both the hedonistic conception as the eudaimonic conception. In other words, well-being is obtained when the worker experiments high levels of positive affection, low levels of negative affection and when he/she perceives his/her expressivity and fulfillment in work.

The instrument reaction to tutor performance presented excellent internal consistency indexes, considered reliable and valid and, therefore, a trustworthy tool for evaluating the satisfaction of the participants regarding tutor performance. Finally, the reaction to institutional procedures scale demonstrated its reliability for evaluating the satisfaction of the participants regarding tutor performance.

In Model A, the regression analyses showed that only the reaction to institutional procedures predicts learning. Thus, healthcare providers that present more positive reactions to institutional procedures are those who present higher grade means in the evaluative activities. França (2009)França, G. (2009). Os ambientes de aprendizagem na época da hipermídia e da educação a distância. Perspectivas em Ciência da Informação, 14(1), 55-65. doi: 10.1590/S1413-99362009000100005
https://doi.org/10.1590/S1413-9936200900...
confirms that the institutional procedures can facilitate learning, demonstrating the importance of the quality of content, didactic material of the educational action and promotion of collaborative activities, such as the forums and chats that allow the improvement of communication and, consequently, corroborate the learning of the participants.

The reaction to institutional procedures can influence learning, especially when the e-learning actions are related to the daily work of the participant, that is, when there is correspondence between the content of the program and the personal objectives of the student (Blackman, Mannix, & Sinclair, 2014Blackman, I. R., Mannix, T., & Sinclair P. M. (2014). Developing renal nurses’ buttonhole cannulation skills using e-learning. Journal of Renal Care, 40(1), 55-63. doi: 10.1111/jorc.12047
https://doi.org/10.1111/jorc.12047...
). The role of the institutional design is also highlighted, which include the contents, language used and links in the promotion of learning (Merchant, Goetz, Cifuentes, Keeney-Kennicutt, & Davis, 2014Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29-40. doi: 10.1016/j.compedu.2013.07.33
https://doi.org/10.1016/j.compedu.2013.0...
).

It is worth noting that the reaction to institutional procedures predicts not only learning, but also the transference of training, performance and results, demonstrating that the favorable perceptions in relation to the distance modality aid in the learning process. Thus, the institutional procedures, when planned, organized and structured, positively influence the results of the educational action (Baldwin, Ford, & Blume, 2017Baldwin, T. T., Ford, J. K., & Blume, B. D. (2017). The State of Transfer of Training Research: Moving toward more Consumer-Centric Inquiry. Human Resource Development Quarterly, 28(1), 17-28. doi: 10.1002/hrdq.21278
https://doi.org/10.1002/hrdq.21278...
).

The results that confirm that the reaction to institutional procedures predicts learning show students performing in planning institutional events and human resources managers of the SUS that distance educational actions must be organized in order to ensure the following aspects: the correspondence between the proposed content and the objectives of the program, the correspondence between the content of the program and the personal objectives of the trainee. Furthermore, the sequence of presentation of the units, language used in the program material, recommended readings, novelties, reminders and links published in the virtual learning environment, activities/tasks proposed at the end of each unit, contents suggested for each unit, hours of study suggested for each unit, forums and chats, exchange of messages between students and the advisors for the solution of errors in the activities/tasks, must be adapted.

In Model B, the regression analysis showed that only the area of professional performance significantly predicts learning. Thus, it is understood that the individuals that work in a certain professional field learn more through the target-program then the professionals that perform in other areas. Some studies show that workers from certain occupational class can involve themselves more effectively with PEH actions, according to Pimmer et al. (2014)Pimmer, C., Brysiewicz, P., Linxen, S., Walters, F., Chipps, J., & Gröhbiel, U. (2014). Informal mobile learning in nurse education and practice in remote areas-A case study from rural South Africa. Nurse Education Today, 34(11), 1398-1404. doi: 10.1016/j.nedt.2014.03.013
https://doi.org/10.1016/j.nedt.2014.03.0...
. Such result confirms that the sociodemographic variables related to the professional profile of the clients can influence trainee learning.

From this study emerge a few practical implications that can give rise to possible orientations to be passed on to professionals performing directly in planning, execution and evaluation of educational actions directed at public healthcare professionals and at education institutions that offer distance graduation programs in the field of healthcare. The promotion of the results of this work can, therefore, update and provide feedback to change actions of the educational process, aiming at ensuring the higher activity of e-learning actions.

Among the methodological limitations of this study, the non-use of more robust and rigorous statistical analyses for testing the relations between TD&E variables that could aid in the development in the theoretical field of institutional evaluation is prominent. Therefore, in addition to EFA and multiple regression analyses, according to Martins (2012)Martins, L. B. (2012). Aprendizagem em ações educacionais a distância: Fatores influentes no desempenho acadêmico de universitários (Master thesis, Faculdade de Filosofia, Ciências e Letras de Ribeirão Preto, Universidade de São Paulo, Ribeirão Preto, SP, Brazil). Retrieved from http://www.teses.usp.br/teses/disponiveis/59/59137/tde-02102012-123253/pt-br.php
http://www.teses.usp.br/teses/disponivei...
, the performance of the confirmatory factorial analysis (CFA) of the instruments and of the structural equation modeling (SEM) is fundamental. However, because of the ratio of low questionnaire feedback and the reduced number of healthcare providers that comprise the sample of the target-program, such analyses were not conducted.

Finally, the impossibility of generalizing the results found for other contexts and other participants is highlighted, given that the investigation model test was conducted only in one distance graduation program in the area of healthcare. Furthermore, the participants of the target-program performed professionally in the most various work organizations executing different activities related to the management of healthcare organizations and the promotion, prevention, recovery and rehabilitation of healthcare for SUS users.

Learning evaluation in DL must also be interesting for the development of further researches. A learning indicator must measure exactly the performance described in the institutional objectives of the educational action, belong to the same objective domain (cognitive, affectionate or psychomotor), and to the same level of complexity (cognitive domain), of internalization (affection domain) or of automation (psychomotor domain) and be applied under the same conditions specified in the institutional objective (Abbad, Borges-Ferreira, & Nogueira, 2006Abbad, G. S., Borges-Ferreira, M. F., & Nogueira, R. (2006). Medidas de aprendizagem em avaliação de TD&E. In J. E. Borges-Andrade, G. S. Abbad, & L. Mourão (Eds.), Treinamento, desenvolvimento e educação em organizações e trabalho: Fundamentos para a gestão de pessoas (pp. 469-488). Porto Alegre, RS: Artmed.). Thus, it is expected that future studies dedicate to the investigation of whether the evaluative activities used in e-learning educational actions are, in fact, reliable indicators of leaning.

Considering that distance educational actions have been frequently used in the context of public healthcare, empirical studies that approach the evaluation of institutional events directed to healthcare professionals generate contributions that can be useful for planning, executing and evaluating this type of educational action. It is expected that the present work contributes in order for new studies to promote the dialog between universities and the SUS regarding important elements for promoting the quality of on-line educational actions.

# References

• Abbad, G. S., Borges-Ferreira, M. F., & Nogueira, R. (2006). Medidas de aprendizagem em avaliação de TD&E. In J. E. Borges-Andrade, G. S. Abbad, & L. Mourão (Eds.), Treinamento, desenvolvimento e educação em organizações e trabalho: Fundamentos para a gestão de pessoas (pp. 469-488). Porto Alegre, RS: Artmed.
• Baldwin, T. T., Ford, J. K., & Blume, B. D. (2017). The State of Transfer of Training Research: Moving toward more Consumer-Centric Inquiry. Human Resource Development Quarterly, 28(1), 17-28. doi: 10.1002/hrdq.21278
» https://doi.org/10.1002/hrdq.21278
• Bell, B. S., Tannenbaum, S. I., Ford, J. K., Noe, R. A., & Kraiger, K. (2017). 100 Years of Training and Development Research: What we know and where we should go. Journal of Applied Psychology, 102(3), 305-323. doi: 10.1037/apl0000142
» https://doi.org/10.1037/apl0000142
• Bhatti, M. A., Ali, S., Isa, M. F. M., & Battour, M. M. (2014). Training Transfer and Transfer Motivation: The influence of individual, environmental, situational, training design, and affective reaction factors. Performance Improvement Quarterly, 27(1), 51-82. doi: 10.1002/piq.21165
» https://doi.org/10.1002/piq.21165
• Blackman, I. R., Mannix, T., & Sinclair P. M. (2014). Developing renal nurses’ buttonhole cannulation skills using e-learning. Journal of Renal Care, 40(1), 55-63. doi: 10.1111/jorc.12047
» https://doi.org/10.1111/jorc.12047
• Borges-Andrade, J. E. (2006). Avaliação integrada e somativa em TD&E. In J. E. Borges-Andrade, G. S. Abbad, & L. Mourão (Eds.), Treinamento, desenvolvimento e educação em organizações e trabalho: Fundamentos para a gestão de pessoas (pp. 343-358). Porto Alegre, RS: Artmed.
• Brandão, H. P., Borges-Andrade, J. E., & Guimarães, T. A. (2012). Desempenho organizacional e suas relações com competências gerenciais, suporte organizacional e treinamento. Revista de Administração, 47(4), 523-539. doi: 10.5700/rausp1056
» https://doi.org/10.5700/rausp1056
• Falola, H. O., Osibanjo, A. O., & Ojo, S. I. (2014). Effectiveness of training and development on employees’ performance and organizations competitiveness in the Nigerian Banking industry. Bulletin of the Transilvania University of Brasov. Economic Sciences, 7(1), 161-170. Retrieved from http://webbut.unitbv.ro/BU2014/Series%20V/BULETIN%20V%20PDF/22_FALOLA%2029.06-MARE.pdf
» http://webbut.unitbv.ro/BU2014/Series%20V/BULETIN%20V%20PDF/22_FALOLA%2029.06-MARE.pdf
• França, G. (2009). Os ambientes de aprendizagem na época da hipermídia e da educação a distância. Perspectivas em Ciência da Informação, 14(1), 55-65. doi: 10.1590/S1413-99362009000100005
» https://doi.org/10.1590/S1413-99362009000100005
• Garavan, T. N., Carbery, R., O’Malley, G., & O’Donnell, D. (2010). Understanding participation in e-learning in organizations: A largescale empirical study of employees. International Journal of Training and Development, 14(3), 155-168. doi: 10.1111/j.1468-2419.2010.00349
» https://doi.org/10.1111/j.1468-2419.2010.00349
• Gotardo, R. A., Souza, H. A., Hruschka, E., Jr., & Viana, D. B. G. (2012). Teorias de aprendizagens na EAD: Fundamentação no uso dos recursos de design instrucional e design interacional. Paper presented at the SIED Simpósio Internacional de Educação a Distância (pp. 1-13). São Carlos, SP. Retrieved from http://sistemas3.sead.ufscar.br/ojs/index.php/sied/article/view/365
• Logan, J. W., Lundberg, O. H., Roth, L., & Walsh, K. R. (2017). The effect of individual motivation and cognitive ability on student performance outcomes in a distante education environment. Journal of Learning in Higher Education, 13(1), 83-91. Retrieved from http://files.eric.ed.gov/fulltext/EJ1139727.pdf
» http://files.eric.ed.gov/fulltext/EJ1139727.pdf
• Manuti, A., Pastore, S., Scardigno, A. F., Giancaspro, M. L., & Morciano, D. (2015). Formal and informal learning in the workplace: A research review. International Journal of Training and Development, 19(1), 1-17.doi: 10.1111/ijtd.12044
» https://doi.org/10.1111/ijtd.12044
• Martins, L. B. (2012). Aprendizagem em ações educacionais a distância: Fatores influentes no desempenho acadêmico de universitários (Master thesis, Faculdade de Filosofia, Ciências e Letras de Ribeirão Preto, Universidade de São Paulo, Ribeirão Preto, SP, Brazil). Retrieved from http://www.teses.usp.br/teses/disponiveis/59/59137/tde-02102012-123253/pt-br.php
» http://www.teses.usp.br/teses/disponiveis/59/59137/tde-02102012-123253/pt-br.php
• Martins, L. B. (2016). Impacto do treinamento no trabalho: Determinantes individuais e contextuais em cursos corporativos a distância (Doctoral dissertation, Faculdade de Filosofia, Ciências e Letras de Ribeirão Preto, Universidade de São Paulo, Ribeirão Preto, SP, Brazil). Retrieved from http://www.teses.usp.br/teses/disponiveis/59/59137/tde-07112016-110039/pt-br.php
» http://www.teses.usp.br/teses/disponiveis/59/59137/tde-07112016-110039/pt-br.php
• Martins, L. B., & Zerbini, T. (2015). Evidências de validade de instrumentos de reações no ensino superior à distância. Psicologia do Desenvolvimento, 15(1), 116-134. doi: 10.5935/1678-4669.20160030
» https://doi.org/10.5935/1678-4669.20160030
• Martins, L. B., & Zerbini, T. (2016). Fatores influentes no desempenho acadêmico de universitários em ações educacionais a distância. Estudos de Psicologia (Natal), 21(3), 317-327. doi: 10.5935/1678-4669.20160030
» https://doi.org/10.5935/1678-4669.20160030
• Meneses, P. P. M., Zerbini, T., & Abbad, G. (2010). Manual de treinamento organizacional Porto Alegre, RS: Artmed.
• Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29-40. doi: 10.1016/j.compedu.2013.07.33
» https://doi.org/10.1016/j.compedu.2013.07.33
• Miccas, F. L., & Batista, S. H. S. S. (2014). Educação permanente em saúde: Metassíntese. Revista de Saúde Pública, 48(1), 170-185. doi:10.1590/S0034-8910.2014048004498
» https://doi.org/10.1590/S0034-8910.2014048004498
• Mira, V. L., Peduzzi, M., Melleiro, M. M., Tronchin, D. M. R, Prado, M. F. F., Santos, P. T., Lara, E. M. S., ...Borges-Andrade, J. E. (2011). Análise do processo de avaliação da aprendizagem de ações educativas de profissionais de enfermagem [Special issue]. Revista da Escola de Enfermagem da USP, 45, 1574-1581. doi: 10.1590/S0080-62342011000700006
» https://doi.org/10.1590/S0080-62342011000700006
• Mourão, L., & Meneses, P. P. M. (2012). Construção de Medidas em TD&E. In G. S. Abbad, L. Mourão, P. P. M. Meneses, T. Zerbini, J. E. Borges-Andrade, & R. Vilas-Boas (Eds.), Medidas de Avaliação em Treinamento, Desenvolvimento e Educação (pp. 50-63). Porto Alegre: Artmed.
• Paschoal, T., & Tamayo, A. (2008). Construção e validação da Escala de bem-estar no trabalho. Avaliação Psicológica, 7(1), 11-22. Retrieved from http://www.redalyc.org/articulo.oa?id=335027183004
» http://www.redalyc.org/articulo.oa?id=335027183004
• Persky, S., Kaphingst, K. A., McCall, C., Lachance, C., Beall, A. C., & Blascovich, J. (2009). Presence relates to distinct outcomes in two virtual environments employing different learning modalities. Cyberpsychology & Behavior, 12(3), 263-268. doi: 10.1089/cpb.2008.0262
» https://doi.org/10.1089/cpb.2008.0262
• Phillips, J. J., & Phillips, P. P. (2016). Handbook of training evaluation and measurement methods (4th ed.). New York: Routledge.
• Pimmer, C., Brysiewicz, P., Linxen, S., Walters, F., Chipps, J., & Gröhbiel, U. (2014). Informal mobile learning in nurse education and practice in remote areas-A case study from rural South Africa. Nurse Education Today, 34(11), 1398-1404. doi: 10.1016/j.nedt.2014.03.013
» https://doi.org/10.1016/j.nedt.2014.03.013
• Prichard, J. S., Bizo, L. A., & Stratford, R. J. (2011). Evaluating the effects of team-skills training on subjective workload. Learning and Instruction, 21(3), 429-440. doi: 10.1016/j.learninstruc.2010.06.003
» https://doi.org/10.1016/j.learninstruc.2010.06.003
• Quaglia, I., Oliveira, A., & Velho, A. P. M. (2015). Capacitação em saúde na Educação a Distância (EAD): Uma análise sistemática do conteúdo. Saúde e Pesquisa, 8, 103-112. Retrieved from http://periodicos.unicesumar.edu.br/index.php/saudpesq/article/view/3767/2516
» http://periodicos.unicesumar.edu.br/index.php/saudpesq/article/view/3767/2516
• Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics New York: Harper Collins College.
• Tims, M., Bakker, A. B., & Derks, D. (2013). The impact of job crafting on job demands, job resources, and well-being. Journal of Occupational Health Psychology, 18(2), 230-240. doi: 10.1037/a0032141
» https://doi.org/10.1037/a0032141
• Vieira, M., & Chinelli, F. (2013). Relação contemporânea entre trabalho, qualificação e reconhecimento: Repercussões sobre os trabalhadores técnicos do SUS. Ciência & Saúde Coletiva, 18(6), 1591-1600. doi: 10.1590/S1413-81232013000600011
» https://doi.org/10.1590/S1413-81232013000600011
• Yanssen, Z., & Khafar, M. (2009). Sales training effectiveness: Managers perceptions in the United Arab Emirates organizations. The Business Review, 13(2), 120-128. doi: 10.1108/MIP-02-2013-0029
» https://doi.org/10.1108/MIP-02-2013-0029
• Zerbini, T., & Abbad, G. S. (2009a). Reação ao desempenho do tutor em um curso a distância: Validação de uma Escala. Estudos e Pesquisas em Psicologia, 9(2), 447-463. Retrieved from http://www.e-publicacoes.uerj.br/index.php/revispsi/article/view/9114/7496
» http://www.e-publicacoes.uerj.br/index.php/revispsi/article/view/9114/7496
• Zerbini, T., & Abbad, G. S. (2009b). Reação aos procedimentos instrucionais de um curso via internet: Validação de uma escala. Estudos de Psicologia (Campinas), 26(3), 363-371. doi: 10.1590/S0103-166X2009000300009
» https://doi.org/10.1590/S0103-166X2009000300009

# Publication Dates

• Publication in this collection
Mar 2019