Evaluation of the online management course from the perspective of former students

ABSTRACT

Objective

To evaluate the online course from the perspective of e-learners as well as the relation between variables.

Method

A quantitative, descriptive and exploratory study.

Results

After three years, the satisfaction rates in the three listed categories presented an average rate higher than 75%. The coefficients indicated a high consistency of the questionnaire. Considering the overall rates in the three years period, the Instructor Performance category presented the highest rate. Strong associations between Self-Assessment and Instructor Performance, Self-Evaluation and Program of the Course and Instructor performance and Program of the course were identified. There was no association between the three categories mentioned with any other variables existing in the study.

Conclusion

E-learners expressed satisfaction with the course that means favored the interaction and the promotion of collective knowledge in nursing management. Also aspects need to be improved, especially the training of the instructor to mediate discussions and encourage student involvement throughout the course.

Education, Distance; Training; Education, Nursing, Continuing; Evaluation

RESUMO

Objetivo

Avaliar curso online na perspectiva do egresso e verificar a relação entre variáveis.

Método

Estudo quantitativo, descritivo e exploratório, aplicado aos participantes no final de três versões de um curso de atualização onlinena temática de Gerenciamento em Enfermagem.

Resultados

Os índices de satisfação nas três categorias elencadas, em três anos, apresentaram resultados acima de 75,0%. Os coeficientes obtidos indicaram alta consistência do questionário. Considerando o índice total, a categoria Desempenho do tutor foi a de índice mais alto. Fortes associações entre Autoavaliação e Desempenho do tutor, Autoavaliação e Programa do curso e Desempenho do tutor e Programa do curso foram identificadas. Não houve associação entre as três categorias referidas com as demais variáveis do estudo.

Conclusão

Os egressos demonstraram satisfação com o curso, que favoreceu a interação e a promoção do conhecimento coletivo no gerenciamento em enfermagem. Foram reconhecidos, também, aspectos que carecem de melhorias, com destaque à capacitação do tutor para mediar discussões e estimular o envolvimento do estudante ao longo do curso.

Educação a Distância; Capacitação; Educação Continuada em Enfermagem; Avaliação

RESUMEN

Objetivo

Evaluar curso online desde la perspectiva de los egresos y verificar la relación entre las variables.

Método

Estudio cuantitativo, descriptivo y exploratorio aplicado a los participantes en el final de tres versiones de un curso de actualizaciónonline en la temática de Gestión en Enfermería.

Resultados

Los índices de satisfacción en las tres categorías enumeradas, en tres años, presentaron resultados arriba de los 75,0%. Los coeficientes indican una alta consistencia del cuestionario. Teniendo en cuenta el índice total, la categoría Desempeño del Tutor ha presentado el índice más alto. Se identificaron fuertes asociaciones entre Autoevaluación y Desempeño del Tutor, Auto Evaluación y Programa del Curso, y Desempeño del Tutor y Programa del Curso. No se identificaron asociaciones entre las tres categorías mencionadas con las otras variables del estudio.

Conclusión

Los egresos expresaron su satisfacción por el curso que favorecía la interacción y la promoción del conocimiento colectivo en la gestión de enfermería. También fueron reconocidos los aspectos que deben ser mejorados, sobre todo la capacitación del tutor para mediar discusiones y estimular la participación de los estudiantes a lo largo del curso.

Educación a Distancia; Capacitación; Educación Continua en Enfermería; Evaluación

INTRODUCTION

The advance of Information and Communications Technologies (ICT) and the increasingly access to them have contributed to social changes in all areas and, thus, in the educational field. The facilitated access to ICT enabled the development of learning styles based on the intensive use of computer and of the web, breaking the geographic-temporal barriers of access to formal and non-formal education(¹).

Distance education entails countless benefits, additionally to geographic-temporal facilities like convenience, easier access to teaching material, dynamic environment, exchange of experiences among agents participating in the process(²).

Such break of barriers makes room for new ways of consumption of information in the educational scenario, mainly marked by the traditional forms of on-site classroom. Similarly, the education of human resources in the field of health and permanent education are mainly conditioned to that traditional scenario, restricted to courses developed in the working environment for a large number of workers, but that hardly promotes interaction among professionals experiencing different working realities.

It is worth mentioning that permanent education in health bears a crucial peculiarity that should be considered, i.e., students are adults, professionals living in a social context where perceptions and education are permeated by experiences in continuous process of transformation and interchange with other professionals or with the population targeted by their work(³).

There is a need for developing more creative learning means to keep pace with the quick scientific development, not disregarding the dynamic context of work in health. ICT-based education in distance courses is an option that enhances continued education, the exchange of experiences, access to scientific knowledge and consistent upgrading.

Online training or course, as it is referred to herein, can be understood as a set of educational actions systematically planned in the light of improving and acquiring knowledge, skills and attitudes by students, with space and time flexibility between professors and students and synchronous and asynchronous activities that facilitate interaction and interactivity among agents(⁴). Those educational actions aim at acquiring competences to fill in the work performance gaps and prepare professionals to take on new roles(⁵).

Distance Education (DE) is not a novelty in Brazil. Throughout years, different educational models have been implemented with some variations and combinations. The most prominent ones are the teleaula,videoaula and web(⁶).

The official concept of DE by the Ministry of Education is (⁷) “an educational modality where didactic-pedagogic mediation in teaching and learning processes takes place through the use of information and communications means and technologies, where students and tutors develop educational activities at different places or times”.

In the last few years, education through internet-based distance courses became very popular in the national and international scenarios, despite the questioning about its quality and validity as educational activity. According to a survey by the Brazilian Association of Distance Education (ABED, in Portuguese) amongst 3.5 million students assessed in 2011, 77% used to take some kind of free course for professional supplementation and upgrading(⁸). In the same survey the evaluation of courses regarding their quality started to be mentioned as one of the main obstacles posed to the DE⁽⁸⁾.

In face of the aforementioned, there is an urgent need for developing and applying consistent mechanisms to evaluate quality and satisfaction regarding the course, mainly by the target audience, i.e., the students. Thus, evaluation aims at observing if the course objectives are being fulfilled, and providing guidance to the team engaged in the course design to find more pertinent forms of dynamic as these provide the information required to trigger changes and promote continuous improvement and upgrading.

Courses can be assessed from different theoretical perspectives. Here, the guiding referential comes from Social and Labor Psychology that adds to the four levels described by Kirkpatrick, the analysis of other labor environment components(⁹) and the articulated analysis of behaviors to provide organizational support and to help the transfer and impact of training on the job(¹⁰).

The opinion of distance course participants is crucial to evidence quality, employing pre-established criteria such as structure, layout and online environment, tutor’s attitude and teaching material(¹¹).

Therefore, the evaluation of courses in the light of former students has driven the realignment of it; thus, this study bears the objectives of evaluating the online Nursing Management course in the light of former students, as well as checking the existence of links between the variables of the study.

METHOD

It is a study of quantitative nature, descriptive and exploratory, applied to participants concluding three versions of an online refreshing course in Nursing Management.

This course was designed in the light of Andragogy – theory of significant learning – and of dialectic methodology facilitated by ICTs with synchronous and asynchronous activities scheduled. Resulted from an international partnership, it was led by professors and tutors of two institutions of higher education in nursing – one in Brazil and one in Portugal – additionally to technical and management tutors. Students were nurses of both nationalities. Because of the course design, activities were mainly online.

The virtual learning environment was the Moodle software (Modular Object-Oriented Dynamic Learning Environment) and, for online classes, it was the Adobe Connect® software that enabled managing and monitoring distance education. There were 90 class hours delivered in 12 weeks, with about 8 weekly hours of activities.

Activities were distributed by topics on nursing management, and aimed at providing support to the nurse’s managerial dimension practice. Additionally to the contents, topics were made up by case studies in group, reading and interactive activities, forums, chats, synchronous classes, among others. Learning evaluation was through case studies and, by the end of the course, an individual case study was applied.

The sampling technique for this survey was non-probabilistic upon agreement on participating by the end of each version of the course. Of the 98 graduates of the three versions, 77 nurses agreed on answering the questionnaire prepared to evaluate students’ satisfaction, after signing the Free and Informed Consent Term (FICT). The survey was approved by the Committee of Ethics in Research of the proposing institution through protocol # 1062/2011/CEP-EEUSP – SISNEP CAAE: 0068.0.196.000-11, and the ethical and legal precepts of research were respected.

Therefore, population was composed by 77 graduates of the three courses delivered from 2011 to 2013. The statistical analysis excluded respondents that have not completely filled in the instrument and, thus, n in each treatment considered only the valid cases.

To gather data, a questionnaire was applied with eight socio-demographic and 52 items to evaluate the course through the Likert scale with 5 agreement levels (from 1 to 5), as shown in Figure 1.

Figure 1
Scale of responses to the course evaluation questionnaire – São Paulo, SP, Brazil, 2013.

The questionnaire was built inspired by the model proposed by the researcher who expanded the reaction evaluation technique, named evaluation of satisfaction as it expresses the participant’s satisfaction in relation to the variables of the three levels of evaluation: reaction, learning and results(¹²).

The 52 items were divided into three theoretical categories of analysis; Self-evaluation with 22 items; Tutor’s performance with 11; and, Course schedule with 19.

Responses were organized in data sheets, using the Excel® software. Data analysis was supported by the Statistical Package for Social Sciences SPSS® software.

Data was submitted to descriptive and inferential statistical analysis, assuming p≤0.05 as significance. The students’ satisfaction index was obtained through data standardization.

The instrument reliability was checked through category of analysis, using the Cronbach Alpha that shows the degree of co-variance of items, pointing out the internal consistence; the highest the reliability coefficient, more internally consistent the measure is (¹³).

Following were the coefficients achieved: Self-evaluation 0.911; Tutor’s performance 0.945; and, Course schedule 0.940, pointing out high consistency of the scale, evidencing its reliability to evaluate the course.

RESULTS

When characterizing students we found that 67.5% of nurses that completed the survey lived in Brazil and 32.5% in Portugal; 89.6% were females. The average age of subjects was 39 years old (SD±9.0). Of the nurses responding the questionnaire 94.8% held formal labor link and worked for 12.61 years (SD±10.3) on average. The prevailing work site was the hospital setting (55.8%), 20.8% in primary care and 5.2% in education institutions. Specialization (59.7%) was the highest education level of respondents followed by undergraduate nursing course (23.4%), while Master’s and Doctors’ degrees accounted for 16.9%.

Of the offices, 40.3% worked as assistance nurse, 39% as unit chiefs, and 5.2% were heading health services.

Regarding the level of computer skills 41% said to be very good, 35.1% said to be good, and 9.1% said to be excellent, an indication that 85.7% of students recognized satisfactory level of computer skills.

The course location was: 62.3% at home; 22.1% on job; and, 7.8% both locations. Only six respondents said they took the course in other site than home and work.

Of the course participants, 63.6% responded it was not their first experience with distance course, although the instrument failed in checking if previous experiences were with online courses or other formats of course.

Tables 1, 2 and 3reported the students’ satisfaction indexes in relation to the course by item evaluated, considering 0 to 100 intervals, the average and standard deviation from 2011 to 2013.

Table 1
Satisfaction index of the Self-evaluation analysis category – São Paulo, SP, Brazil, 2013.

Self-evaluation	Average	SD
1. The course has met my needs of learning on the topic.	75.00	22.213
2. The course provided me with new knowledge.	87.99	18.412
3. My interest on the subject has increased after taking the course.	83.44	20.524
4. Using the computer to take the course was easy.	84.09	21.038
5. I have frequently resourced to the course “Library”.	64.47	26.224
6. I can use in my work place what I have learned in the course.	73.70	22.542
7. This course provided me with opportunities to exchange valuable experiences with other participants.	72.73	23.711
8. I felt free to disagree with the ideas presented by professor(s).	79.93	19.587
9. I felt encouraged to discuss ideas on the course topic with other participants.	75.97	21.628
10. The course contributed to my professional work.	75.66	22.721
11. I have organized my time to take the course with at ease.	68.51	25.132
12. I have regularly followed the messages sent.	85.39	19.597
13. I felt encouraged to make deeper search on the topic after the course.	78.25	19.597
14. My previous knowledge on the matter facilitated my learning.	71.75	21.210
15. I tried to read the bibliography recommended by the professor, even the non-mandatory.	73.03	26.065
16. I have delivered the papers required on the deadline.	87.01	25.524
17. I liked taking the “distance” course.	83.44	24.539
18. By the end of the course I achieved the expected performance according to the objectives.	75.97	22.741
19. I have taken the best of the course.	79.22	20.839

Table 2
Satisfaction index of the Tutor’s performance analysis category – São Paulo, SP, Brazil, 2013.

Tutor’s performance	Average	SD
1. Presented the objectives of the topics in a clear way.	84.74	18.207
2. Clarified the participants’ doubts in a timely way.	86.04	19.234
3. Sparked my interest on the topic.	79.22	20.839
4. Respected opinions contrary to those s/he presented.	85.00	19.690
5. Showed to master the content.	91.23	17.094
6. Used updated references on the matter.	89.29	17.410
7. Presented examples that facilitated understanding the subject.	86.51	18.458
8. Summarized the main ideas approached in class.	85.86	18.857
9. Encouraged us to seek additional information on the matter.	81.49	21.614
10. Encouraged critical analysis.	84.09	19.830
11. There was interaction with tutors, professors and students.	86.69	18.842

Table 3
Satisfaction index of the Course schedule category – São Paulo, SP, Brazil, 2013.

Schedule	Average	SD
1. The equipment I used was compatible with the course requirements.	91.56	17.495
2. The course objectives were clearly presented on the instruction pages.	91.56	16.529
3. The language employed was accessible.	84.74	19.515
4. The content was compatible with the class hours scheduled to the course.	78.25	21.971
5. The course was structured in such a way as to keep me motivated until the conclusion.	77.60	21.683
6. The course content was updated.	87.99	18.412
7. The course content was adequate to reach the objectives proposed.	83.44	20.921
8. The content sequence has facilitated understanding the subject.	85.06	18.699
9. The course content was relevant to perform my roles at work.	78.62	24.733
10. Access to the course information was easy.	85.39	19.597
11. The “Library” offered relevant materials to have good performance in the course.	80.59	21.832
12. The tutor has clarified doubts about how the course worked.	90.91	17.635
13. It was easy to subscribe to the course.	92.43	17.804
14. The format of the pages facilitated understanding the content of instructions.	88.64	18.383
15. Notices on date of delivery of papers have facilitated meeting the deadlines.	84.09	23.265
16. The discussion list has fostered debates that helped my learning.	71.67	26.102
17. E-mail messages provided relevant information.	86.69	20.910
18. Time assigned for performing tasks was enough.	72.40	24.192
19. The tasks requested were coherent with the objectives of the modules.	87.34	18.417

In the self-evaluation category, of the 19 propositions evaluated 13 (68.4%) reached 75% as minimum satisfaction degree. The best indexes were found for items on acquisition of new knowledge after the course (87.99); delivery of papers on the deadline (87.01); and, regular follow-up of messages sent by the course teaching staff (85.39). On the other hand, the lowest indexes were regarding the students’ consultation to course library (recommended reading) (64.47), organization of time to take the course at ease (68.51), and previous knowledge facilitating the learning process (71.75).

Regarding the tutor’s performance analysis, all propositions evaluated reached the minimum satisfaction level of 80%, except for the proposition regarding tutor’s capacity of sparking interest on the topic (79.22).

The propositions with best averages in the category of course schedule were easiness of subscribing to the course (92.43), if the equipment used by the student was responsive to that required by the course (91.56) and if the course objectives presented in instructions were clear (91.56). In the same category, the worst averages were found in the following items: if propositions in the discussion list have fostered the debate and assisted learning (71.67); if time assigned to perform the tasks proposed was enough (72.40), and if the course was structured in such a way as to keep students motivated until the conclusion (77.60).

Table 4 presents the students’ satisfaction indexes with the course, by year and by category of analysis.

Table 4
Students’ satisfaction index with the course, by year of the course and category of analysis – São Paulo, SP, Brazil, 2013.

		N	Average	Standard Deviation Lower threshold	Interval of Confidence – 95%
					Upper threshold
Self-evaluation	2011	22	79.3388	11.19263	74.3763	84.3014
	2012	24	82.0549	9.07842	78.2214	85.8884
	2013	27	72.7694	16.17068	66.3725	79.1663
	Total	73	77.8020	13.17355	74.7284	80.8756
Tutor’s performance	2011	23	88.3399	13.07926	82.6840	93.9958
	2012	22	90.1860	9.92857	85.7839	94.5880
	2013	25	80.5455	19.56262	72.4704	88.6205
	Total	70	86.1364	15.35673	82.4747	89.7980
Course schedule	2011	23	86.0412	10.66549	81.4291	90.6533
	2012	25	86.8947	9.25335	83.0751	90.7143
	2013	25	80.7368	19.80209	72.5629	88.9107
	Total	73	84.5169	14.20118	81.2036	87.8303

We have observed that indexes for three years and three categories presented results above 70.5% and that the lowest indexes were found for 2013. Considering the total index, the Tutor’s performance category reached the highest index.

For the investigation on association between variables, we could observe there was no association between the satisfaction index averages and the following variables: sex, age, country of residence, schooling level, degree of computer skills, site where the course was taken (home, work, others), if the student had taken distance course before, time of professional exercise, office and work place.

In the categories of analysis, the Pearson’s correlation coefficient displayed strong associations between: Self-evaluation and Tutors’ performance (0.819); Self-evaluation and Course schedule (0.787); Tutor’s performance and Course schedule (0.882).

When comparing categories by year of the course, we found that in 2012 only the Self-evaluation category (ANOVA=0.032) presented average significantly higher than in 2013 (p=0.03).

DISCUSSION

The reaction evaluation tools should approach instructional and administrative aspects of the training program, as these aspects are sensitive to the participants’ reactions. Most of the reaction evaluation instruments prioritize the adoption of close-ended questions, and could result in immobility regarding the perception of unforeseen and particular occurrences in the training. On the other hand, it increases the reliability of results as they allow the re-use and replication of the instrument in different situations (¹⁴).

The results of a training course should produce favorable reactions among participants (¹⁰), and this was recognized in the course evaluated.

This study found no association between satisfaction index and the variables selected, similarly to the finding in a satisfaction survey with web-based course that showed no link between satisfaction and learning with gender, age, learning styles, time spent with the course, interactions, course activities and asynchronous conferences. However, it pointed out that those more skilled in computer were more satisfied with the course, although it has not influenced on the student’s successful completion of the course(¹⁵).

Self-evaluation showed that new knowledge was acquired after the course. Comparative surveys between online and on-site courses were not unanimous regarding significant differences in relation to both kinds of course, as both can lead to changes on behavior and gain of knowledge (¹⁶). Some studies point out better performance in the carrying out of activities by those learning in virtual environments, either through blended learning or e-learning (¹⁷). Other study has ratified the conclusion that clinical evidence-based decisions made by physicians attending online course were more likely to happen than those with no access to interactive activities (¹⁸).

A recent systematic review recognizes online nursing education as an alternative to education; however, no statistical difference was found between on-site and online education groups regarding impact, knowledge, skill and satisfaction. Of the 11 articles selected, four disclosed improvement associated to online education compared to the traditional learning techniques, although differences were not statistically significant. A study disclosed weak impact on the online education capability, although with no significant statistical difference(¹⁹).

The tutor’s performance reached the highest satisfaction indexes; a positive correlation between this category and the self-evaluation category was also found, meaning that the highest satisfaction regarding the tutor’s performance, better the student’s evaluation regarding their performance. A study points out that tutors’ attitude entails significant effects on the satisfaction perceived by students. Tutors have played a core role in the learning process in online environments; when students perceive positive attitude in relation to the tutor there is significant improvement in relation to the student’s satisfaction(¹¹).

Regarding the index interest on the topic sparked by tutors, as this topic achieved the lowest index in the tutor’s performance category, it is understood that tutor’s education should be continuous, including the use of new technologies. The professors’ pedagogical practice assumes continued pursuit for methodologies to nourish critical, creative and transformer students, what presupposes a provoking educator that proposes new ways of learning and of getting adjusted to technologies (²⁰-²¹).

Likewise, the distance education purpose is to let students gradually develop self-suggestion skills, where professor is the mediator of the knowledge-building process (²¹). Literature consistently points out three important elements to the satisfaction perceived by teachers delivering online course in relation to students, tutors and the institution. Student is the most important factor mentioned regarding the influence of satisfaction, leading us to believe that in that light students play a core role in online learning (²²).

The course objectives were presented to students in a clear way, in line with literature, as it enables students to gradually evaluate if objectives are being achieved. Moreover the statement of educational objectives and skills to be developed is crucial to build a comprehensive evaluation process (²³).

In general, results point out that organization, scope, design, linkage and materials used are satisfactory to students. Therefore, it ratifies the findings of surveys that point out that the pleasant interface and simple handling of synchronous and asynchronous communication tools of the course facilitate interaction among students, content and professor, thus helping students in the performance of the activities scheduled (²¹).

The degree of computer skills of former students was a favorable characteristic for online education in this course. Corroborating a survey that evaluated the pedagogical and psychological barriers in online university education, technology was not recognized as barrier in online courses, electronic study materials and learning activities were considered as mild barrier, and barriers related to communications between fellows and tutor were considered as significant obstacles (²⁴), and this could be considered yet another positive aspect of this course, as the index about interaction among tutors, professors and students was evaluated as satisfactory.

It is worth mentioning that the course was based on the premise of building a collaborative online learning room aimed at social interaction, maximizing the interchange of ideas, knowledge and experiences between students-students and students-tutors of both nationalities. Also considering that in this study students are nurses with peculiar professional experiences, the approximation of students mediated by the tutor enhanced the discussion based on the theoretical fundamentals of nursing management, seeking for solutions to the everyday issues found in nurses’ managerial work.

CONCLUSION

The former students’ satisfaction index showed that the course has generally met the requirements described in literature for online courses, and also recognized aspects that should be improved, notably the tutor’s training to mediate discussions and encourage the engagement of students throughout the course.

The association among the three categories of analysis, namely Self-Evaluation and Tutor’s performance; Self-evaluation and Course schedule; and, Tutor’s performance and Course schedule has disclosed the interdependence among these, ratifying the intrinsic link among the evaluation variables. Therefore, we could state that tutor’s performance and the course schedule categories, under the responsibility of the course supporting team, have influenced on former students’ satisfaction with tutor’s performance.

No links between socio-demographic variables and satisfaction index were found in the three categories of analysis.

The evaluation tool is not the focus of this study but, to provide reliability to the course evaluation, we observed internal consistency and accuracy in the three categories of analysis, although being tested in a population shorter than that established for validation, which could be considered to be a limitation of the study. The delivery of further courses may enable the confirmatory factorial analysis and a new test of reliability.

The results achieved allow us to conclude that the course has enabled the development of a virtual community of practice and learning in cyberspace, favoring the interaction and collaboration in the building, promotion and application of collective knowledge in management in nursing.

REFERÊNCIAS

• ¹
  Laguardia J, Casanova A, Machado R. A experiência de aprendizagem on-line em um curso de qualificação profissional em saúde. Trab Educ Saúde. 2010;8(1):97-122.
• ²
  Ribeiro EL. Uma avaliação da importância da gestão da qualidade na prática da educação à distância [dissertação]. São Paulo: Instituto de Ciências Exatas e Tecnológicas, Universidade Estadual Paulista; 2010.
• ³
  Struchiner M, Roschke MA, Ricciardi RMV. Formação permanente, flexível e a distância pela internet: curso de gestão descentralizada de recursos humanos em saúde. Rev Panam Salud Pública. 2002;11(3):158-65.
• ⁴
  Zerbini T, Abbad GS. Impacto de treinamento no trabalho via internet. RAE Eletr [Internet]. 2005 [citado 2014 ago. 06];4(2). Disponível em: http://www.scielo.br/pdf/raeel/v4n2/v4n2a01.pdf
  » http://www.scielo.br/pdf/raeel/v4n2/v4n2a01.pdf
• ⁵
  Abbad GS, Borges-Andrade JE. A aprendizagem humana nas organizações de trabalho. In: Zanelli JC, Borges-Andrade JE, Bastos AVB, editores. Psicologia, organizações e trabalho no Brasil. Porto Alegre: Artmed; 2004. p. 237-75.
• ⁶
  Moran JM. Modelos e avaliação do ensino superior a distância no Brasil. Rev ETD Educ Temática Digital [Internet]. 2009 [citado 2014 ago. 06];10(2). Disponível em: https://www.fe.unicamp.br/revistas/ged/etd/article/view/2004
  » https://www.fe.unicamp.br/revistas/ged/etd/article/view/2004
• ⁷
  Brasil. Decreto n. 5.622, de 19 de dezembro de 2005. Regulamenta o art. 80 da Lei 9.394, de 20 de dezembro de 1996, que estabelece as diretrizes e bases da educação nacional [Internet]. Brasília; 2005 [citado 2014 Ago 06]. Disponível em: http://www.planalto.gov.br/ccivil_03/_ato2004-2006/2005/Decreto/D5622.htm
  » http://www.planalto.gov.br/ccivil_03/_ato2004-2006/2005/Decreto/D5622.htm
• ⁸
  Associação Brasileira de Educação a Distância. Censo EAD.BR: Relatório Analítico da Aprendizagem a Distância no Brasil, 2011 [Internet]. São Paulo: Pearson Education do Brasil; 2012 [citado 2014 set. 24]. Disponível em: http://www.abed.org.br/censoead/censo2012.pdf
  » http://www.abed.org.br/censoead/censo2012.pdf
• ⁹
  Borges-Andrade JE. Avaliação somativa de sistemas instrucionais: integração de três propostas. Tecnol Educ. 1992;11(46):29-39.
• ¹⁰
  Abbad GS. Um modelo integrado de avaliação de Impacto do Treinamento no Trabalho – IMPACT [tese doutorado]. Brasília: Universidade de Brasília; 1999.
• ¹¹
  Sun PC, Tsai RJ, Finger G, Chen YY, Yeh D. What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Comput Educ. 2008;(50):1183-202.
• ¹²
  Mira VL. Avaliação de programas de treinamento e desenvolvimento da equipe de enfermagem de dois hospitais do município de São Paulo [tese livre-docência]. São Paulo: Escola de Enfermagem, Universidade de São Paulo; 2010.
• ¹³
  Pasquali L. Psicometria: teoria dos testes na psicologia e na educação. 5ª ed. Petrópolis: Vozes; 2013.
• ¹⁴
  Pilati R, Borges-Andrade JE. Construção de medidas e delineamento em avaliação de TD&E. In: Borges-Andrade JE, Abbad GdS, Mourão L, editores. Treinamento, desenvolvimento e educação em organizações de trabalho: fundamentos para gestão de pessoas. Porto Alegre: Artmed; 2006. p. 359-84.
• ¹⁵
  Hong KS. Relationships between student’s and instructional variables with satisfaction and Learning from a web-based course. Intern Higher Educ. 2002;5(3):267-81.
• ¹⁶
  Fordis M, King JE, Ballantyne CM, Jones PH, Schneider KH, Spann SJ, et al. Comparasion of the instructional efficacy on internet-based CME with live interactive CME workshops. JAMA. 2005;294(9):1043-51.
• ¹⁷
  Thomson Job Impact Study: the next generation of corporate learning. London: Thomson; 2002.
• ¹⁸
  Casebeer L, Engler S, Bennett NN, Irvine M, Sulkes D, DesLauriers M, et al. A controlled trial of the effectiveness of internet continuing medical education. BMC Med 2008;6:37. doi: 10.1186/1741-7015-6-37.
• ¹⁹
  Lahti M, Hätönen H, Välimäki M. Impact of e-learning on nurses’ and student nurses knowledge, skills, and satisfaction: a systematic review and meta-analysis. Int J Nurs Stud. 2014;51(1):136-49.
• ²⁰
  Weber MAL, Behens MA. Paradigmas educacionais e o ensino com a utilização de mídias. Rev Intersaberes. 2010;5(10):245-70.
• ²¹
  Rangel EML, Mendes IAC, Cárnio EC, Alves LMM, Crispim JA, Mazzo A, et al. Evaluation by nursing students in virtual learning environments for teaching endocrine physiology. Acta Paul Enferm. 2011;24(3):327-33.
• ²²
  Bolliger D, Wasilik O. Factors influencing faculty satisfaction with online teaching and learning in higher education. Distance Educ. 2009;30(1):103-16.
• ²³
  Alavarce DC. Desenvolvimento e avaliação da reação, aprendizagem e impacto de treinamento on-line para profissionais de saúde [tese de doutorado]. São Paulo: Escola de Enfermagem, Universidade de São Paulo; 2014.
• ²⁴
  Simuth J, Sarmany-Schuller I. Principles for e-pedagogy. Procedia Soc Behav Sci. 2012;46:4454-56.

Publication Dates

• Publication in this collection
  Dec 2015

History

• Received
  30 Nov 2014
• Accepted
  11 May 2015