Evaluation in distance learning: a case report of the UNASUS/UERJ postgraduate course in elderly health

Abstract

Objective:

this analytical case study aimed to evaluate the effectiveness of the UNA-SUS/UERJ Specialization Course in Elderly Health.

Method:

a mixed-approach evaluation was used, employing the four levels of the Kirkpatrick evaluation model as a reference.

Results:

the UNA-SUS/UERJ Elderly Health Specialization course presented indicators of success in the three proposed levels of the Kirkpatrick evaluation. The variables identified in the study may be related to a change in the practices of health professionals.

Conclusion:

the evaluation of distance learning activities in Brazil is still a developing area, a finding that is of increasing importance as large-scale distance learning specialization courses are established and require financial resources whilst lacking a proposed approach to evaluation. It is believed that the present study can contribute to the development of new evaluation projects in distance learning, especially those that impact the elderly population in Primary Health Care, supporting new distance learning projects in the area of ​​aging and correcting the direction of current initiatives.

Keywords:
Education; Distance; Education, Graduate; Primary Health Care; Health of the Elderly; Educational Measurement

Resumo

Objetivo:

este estudo de caso analítico teve o objetivo de avaliar a eficácia do Curso de Especialização em Saúde da Pessoa Idosa da UNA-SUS/UERJ.

Método:

utilizou-se uma avaliação com abordagem mista tendo como referencial teórico os quatro níveis de avaliação de Kirkpatrick.

Resultado:

o curso de especialização em saúde da pessoa idosa da UNA-SUS/UERJ apresentou indicadores de sucesso relativos aos três níveis de avaliação de Kirkpatrick. As variáveis identificadas no estudo podem estar relacionadas a uma mudança na prática do profissional de saúde.

Conclusão:

a avaliação de atividades de EaD no Brasil ainda é uma área em desenvolvimento e tal fato ganha relevância à medida que são desenvolvidas especializações a distância em larga escala, demandando recursos financeiros sem uma proposta de abordagem avaliativa. Acredita-se que este trabalho contribua com o desenvolvimento de novos projetos de avaliação em EaD especialmente nos que demonstrem impacto para a população idosa na Atenção Primária à Saúde, subsidiando novos projetos na EaD na área do envelhecimento.

Palavras-chave:
Educação a Distância; Educação de Pós-Graduação; Atenção Primária à Saúde; Saúde do Idoso; Avaliação Educacional

INTRODUCTION

The care of frail elderly persons is a field of particular importance in the context of population aging, and the impairment of their autonomy and independence demands great attention from health professionals. Dealing with this issue involves actively seeking out at-risk or frail elderly individuals, requiring an extended perspective of care¹.

The National Health Policy for the Elderly (PNSPI) established by Ordinance GM/MS Nº 2.528 dated October 19, 2006, seeks to guide individual and collective measures within the Unified Health System (SUS), re-adapting pre-existing strategies and programs to established guidelines for the care of the elderly. It places the responsibilities for the Family Health Strategy (FHS) within the coordination of the intersectoral care that should be given to the elderly population².

The breadth and complexity inherent in aging involve multiple dimensions. These are not immutable, and an intervention in one or more can bring about changes in the configuration of the others, much like in a Rubik’s or "Magic” cube as it is popularly known, where a change in one of the 26 cubes automatically influences the positioning of the others³.

Another challenge arising from this heterogeneity is the need for unique responses based on the identified needs. This requires training and support for health professionals, especially taking into account the creation of strategies of Permanent Education in Health (PEH) in a more extended manner^4,5.

Distance Learning (DL) plays an increasingly important role in attempts to reach more people and reduce costs^6,7. It has proved an outlet for the promotion of PEH in the SUS, due to its decentralization and the need to approach the development of specific skills in areas of major need in an integral and equitable manner.

As a way of developing PEH strategies for health professionals in the SUS, UNA-SUS employs DL as a teaching modality via new Information and Communication Technologies (ICT)⁸. The permanent education strategy of Primary Health Care (PHC) professionals in elderly health was supported by the partnership of three universities, enabling the creation of the Specialization Course in Elderly Health: the Universidade do Estado do Rio de Janeiro (UERJ), the Universidade Federal do Maranhão (UFMa) and the Universidade Federal do Ceará (UFC). The institutions offered 1,500 places on a specialization course in Elderly Health, distributed equally among the three, representing 390 teaching hours. Such courses are non-mandatory for PHC professionals in the FHS, traditional, Family Health Support Centres (FHSC) and management modalities. These activities aimed to allow students to analyze their practice, through tasks and availability of content in the Virtual Learning Environment (VLE), tutor mediation in chatrooms and discussion forums.

The lack of evaluation activities in DL, which fail to accompany the structure of initiatives in this area and are restricted to the intrinsic assessments of students, has already been described in literature⁹. The process of evaluation of a given EAD strategy must be carried out in a non-linear, critical-reflexive manner using quantitative and qualitative criteria according to the identified objectives¹⁰.

Some models, such as the evaluation of the four levels of the Kirkpatrick Model¹¹, may be useful to systematize the most appropriate evaluation process for DL activities, particularly in terms of determining the appropriation of knowledge in the practices of the health professional. According to Waddill¹¹, the methodology of the four levels of the Kirkpatrick evaluation, possesses an advantage over other methods studied as it has already been recognized in literature to evaluate the effectiveness of DL, and offers a flexible method of evaluation, focused on the autonomous construction of knowledge. Kirkpatrick's four levels of evaluation provide a systemic perspective necessary for the analysis of an educational initiative, especially when dealing with a large group representing part of a specific decision-making process⁹.

The four levels of Kirkpatrick's evaluation modified by Waddill¹¹ can be summarized as follows:

• Level 1: Reaction - the perception of the student about the approach to learning offered by the course;

• Level 2: Learning - accumulation of student knowledge with the course;

• Level 3: Behavior - use and form of use of that learned within the course;

• Level 4: Results - the application of the learning to the environment.

It should be noted that each of the Kirkpatrick levels of evaluation involves a cumulative effect in relation to the previous level in the evaluation of the efficiency of the proposed educational model ⁹.

This article aims to offer an analytical case study using a mixed approach to evaluate the efficacy of the UNA-SUS/UERJ Elderly Health Specialization Course using the Kirkpatrick model.

METHOD

The chosen methodology used a mixed approach, with documentary analysis and evaluation of the final report of the course, based on semi-structured and structured questionnaires completed anonymously by the students, and the keywords of the intervention projects. A total of 444 students were enrolled in the Elderly Health Specialization course, with 319 finishing the course, and 299 keywords were identified from the projects of intervention completed by the students at the end of the course.

The Kirkpatrick four-level model was used as a method of analysis, and the first three levels of analysis were addressed as the variables studied are directly related to the students. The first level involved the analysis of the narratives of the 444 students from the course, with a total of 2,643 comments collected from evaluations made at the end of each module and at the end of the course. All the comments were categorized according to the aspect of the course under evaluation, and more than one aspect could be raised by each narrative, resulting in 2,773 statements for analysis. The variables included positive and negative aspects that could be both inherent or not to the course, as well as suggestions made about such aspects and the carrying out of self-assessment. These variables can be termed as endogenous or exogenous to the course. Among the endogenous variables are technical issues and matters relating to content, material, methodology, tutoring and difficulty with tasks. The exogenous variables identified were related to internet access, personal issues and the expressions of opinions about the course.

Summative assessments, pass rate, and student adherence after the first month of the course were analyzed in the second level of evaluation.

The third evaluation level involved the surveying of keywords from the 299 intervention projects of the students, based on the assumption that such projects emerged from the real life practical situations of the students, in which that learnt during the course allows the students to access and internalize the knowledge they have acquired as they propose an intervention for problems identified in their professional reality. A total of 664 identified keywords were categorized, excluding the terms "health", "elderly health" and "elderly" as these correspond to the specific nature of the area in question. In order to identify the degree of relevance of the categorized words, a visualization strategy based on the repetition of the same in the projects was applied, followed by graphic visualization in a tag or word cloud created using the Wordle^TM software package. This feature selects the words that appear most frequently in a text or set of texts, assigning them prominence in size and color.

RESULTS

The results aim to consider Kirkpatrick's first three levels of evaluation through the analysis of questionnaires applied at the beginning and during the course, as well as the pass rates and adherence of the students, ending with an analysis of the degree of relevance of the keywords identified in the intervention projects.

Analysis of Kirkpatrick’s first level of evaluation

The analysis of the 2,643 student narratives resulted in 2,774 statements for the evaluation of the endogenous and exogenous aspects of the course (Table 1), as well as the self-evaluations of students, which provided 177 positionings, and the presence of suggestions for the course, which represented 193 positionings.

Table 1
Evaluations of students regarding the Specialization Course in Elderly Health

Variables	Negative Aspects n (%)	Positive Aspects n (%)
Exogenous Aspects
Opinions of course	1 (0.01)	569 (38.49)
Internet access	35 (3.80)	0 (0.00)
Difficulty with Tasks	223 (24.10)	0 (0.00)
Personal issues	352 (38.10)	0 (0.00)
Endogenous Aspects
Material	2 (0.03)	156 (10.56)
Tutoring	33 (3.60)	68 (4.60)
Methodology	47 (5.10)	211 (14.28)
Technical problems	56 (6.20)	0 (0.00)
Content	177 (19.24)	474 (32.07)
Total	926 (100.00)	1478 (100.00)

Evaluations at the end of each module, which included assessment of the activities of the tutors, were also provided by students during the course through a structured questionnaire that analyzed 19 issues related to tutoring (Figure 1).

Figure 1
Evaluation of tutor performance by students from the specialization course in elderly health.

Analysis of Kirkpatrick’s second level of evaluation

A total of 511 students were enrolled on the course, of whom 67 (13.1%) did not complete the first month or did not access the platform. Another 125 students (24.5%) dropped out sometime after the first month. Of the 319 students who completed the course, 20 (6.27%) failed and 299 (93.73%) completed a final paper based on their own reality, therefore representing project of intervention.

Therefore the retention/permanence rate of the elderly health specialization course was 71.85%, with a pass rate of 93.73%.

Adherence on the course by professional category was also studied, as there were disparities in the number of enrolled students by profession. Figure 2 shows the permanence rate of each professional type, with medicine, social services, pharmacy and nursing achieving adherence rates below the course rates.

Figure 2
Adherence percentage of specialization course in elderly health by profession.

Analysis of Kirkpatrick’s third level of evaluation

The type of use and understanding of the material learnt on the course was measured by surveying the intervention projects and their themes, analyzing the title, objectives and keywords of each such project. A total of 67.55% of the objectives involved actions aimed at the elderly population, 23.74% focused on health professionals involved in elderly care, and 8.69% were aimed at management policies for the elderly population.

After the survey and categorization of the keywords of the 299 intervention projects, graphic visualization of the degree of relevance was carried out via a tag cloud based on number of repetitions in the projects. A total of 664 keywords from the intervention projects corresponded to the main topics covered in the course and the working reality of the health professionals (Figure 3).

Figure 3
Keywords from intervention projects of students of specialization course in elderly health based on degree of relevance.

DISCUSSION

Distance Learning (DL) provides benefits to the teaching-learning process by reducing the distance between teachers and students and providing access to new educational technologies which were not previously widely available. As a result, it allows ownership of knowledge in a collaborative and cooperative manner⁶.

There remain barriers to be overcome such as access to and use of ICT, as well as the DL tools themselves, which are often not known by professionals who have never experienced distance training¹², corroborating the variables used in the present study.

The case study model for the evaluation of the course in elderly health is supported by literature, as it represents the study of a phenomenon that is contemporary to a real given context, the boundaries of which are not clearly defined¹³. In a mixed model using qualitative and quantitative variables, there is an emphasis on studies with non-oppositional qualitative and quantitative aspects, while in the qualitative models it is possible to evaluate social constructions involved in the process which influence the final result¹⁴.

The evaluation carried out by students throughout the specialization course in elderly health offers a first impression of the main problems and benefits of the course. It was noticed that the positive aspects of the course (61.50%) predominated over the negative aspects described (38.50%). Of the negative aspects most frequently mentioned by students, there was a predominance of difficulties with tasks and their deadlines, in addition to an excess of proposed content, followed by personal problems that prevented or made difficult the progress of the student.

Of the positive issues, students expressed positive opinions (38.50%) but did not provide more specific information. Some specific aspects were mentioned, however, such as good quality of content (32.07%), methodology (14.28%) and material (10.56%). Positive evaluations may reflect the variety of methodologies and materials available to students in discussion forums, chatrooms and the exchange of information and guidelines by email. Themes related to the skills developed by the students were accompanied by activities that sought to encourage a critical approach to the knowledge accumulated through analysis of working practice.

One important finding in the narrative of the students was great difficulty with deadlines over extended public holidays, with descriptions of problems of access to the Internet, the organization of time and a perception of an excess of tasks with short deadlines.

This model of evaluation corresponds to studies of dropout and its causes¹⁵. The systemic evaluation of the causes of dropout reveals it is related to endogenous and exogenous factors. Evaluating the biopsychosocial dimension of the students themselves, the negative aspects that contribute most to dropouts from the course coincide with those most frequently reported by students, such as health problems, family problems and difficulty organizing time. These variables were categorized as personal questions. Among the aspects endogenous to the course identified, such as methodology, content, material and mentoring, there was a predominance of positive evaluations, which could directly influence student retention/permanence on the course, as demonstrated by Kirkpatrick's second level of evaluation.

The longitudinal evaluation of tutoring that took place in all modules provided interesting findings. Tutors are key players in DL strategies as they act as facilitators by stimulating the critical-reflexive learning of students¹⁶. The classification of good or excellent performance in 91% of the answers to the 19 assessment questions did not correspond to the majority of the narratives, probably due to differences in relation to the type of questionnaire. There was no guiding question about the work of the tutor in the student narratives, while the specific evaluation of the structured questionnaire meant that the answers were more specific to the variables being studied. It is possible to relate the positive evaluations obtained in the structured questionnaire with the monitoring of the work of the students and the composition of the tutorial body, providing answers directed at the reality of the students, strengthening the educational relationship^16,17.

Aspects considered endogenous or inherent to the course, such as tutoring, should be analyzed throughout its duration to allow the correction of approaches during the pedagogical process, which would also result in lower dropout rates and greater adhesion/permanence¹⁵. The evaluations carried out in each module by the students not only allowed the correction of possible mistakes observed in the didactic-pedagogical process, but also allowed the development of activities aimed at the common reality of the students themselves.

In studies related to the second level of evaluation, the concept of adhesion/adherence, dropout and retention/permanence should be considered. Adherence is considered part of an initial vision regarding the course while retention/permanence is more appropriate to the longitudinal evaluation of the student in a given initiative, both influenced by the biopsychosocial factors of the students themselves, such as their life history and ability to adapt. The complexity of this evaluation is in the connection of the aspects developed during a didactic-pedagogical initiative in a given institution, that is, it makes the concept extremely dependent on the adopted strategy¹³.

It is also worth noting the need to study dropout at various periods in a given DL initiative. It is noticed that in the first weeks of a DL course, especially in the orientation weeks of online classes, the dropout rate of the students is twice that of face-to-face classes, although the variables that directly affect the decision of students are not identified¹⁵. The same difference between DL and face-to-face courses does not occur over the duration of the course, however. This model agrees with one of the definitions of dropout raised in a review of the topic, where it is defined as a student abandoning the course before its completion, despite having acquired knowledge or having reached his or her personal goals¹⁸. This definition leads to questions about how long it takes to acquire some degree of knowledge in a DL initiative. It should be remembered that the first weeks of a course involve issues relating to DL itself as well as agreements on activities, knowledge and the group setting and even adaptation to ICT. In this way, it is important to consider the degree of dropout after the first month of activities.

Studies show a dropout rate of around 57.80% for courses with a similar target audience, as well as dropout rates of around 49.00% to 58.00% in postgraduate courses with more than one stage. It should be noted that courses with collective construction spaces in the form of team meetings when carrying out activities have even lower dropout rates, of around 19.7%¹⁹.

When compared with studies of dropout rates in specialization courses in family health employing face-to-face and distance modalities, the rates of the specialization course in elderly health remained below the average (28.15%), sometimes reaching half that observed in the literature¹⁹.

The analysis of the third level of evaluation, identifying changes in the behavior of the health professional when caring for the health of the elderly, was measured by identifying the themes contained in the intervention projects. The elderly population was the target of 67.55% of the planned actions, while the health professionals themselves appeared in 23.74%, representing a multifaceted and interprofessional approach as shown by the words contained in the tag cloud.

The change in practice observed through the intervention projects translates into a new paradigmatic vision in the educational process with changes in health strategies, aimed at a transformation of reality through the re-signification of know-how and learning in health, resulting in improvements in the SUS itself²⁰.

The use of innovative tools for content evaluation has already been described in literature, such as the use of word clouds for the qualitative analysis of a digital portfolio, especially in the pedagogical training of preceptors in the area of health in the residence programs of the Hospital Universitário Pedro Ernesto²¹. Some studies have used word clouds to evaluate the titles of scientific papers, identifying the most prominent terms and excluding words that form part of a certain context²². This analysis strategy, together with quantitative aspects, allowed the evaluation of the changes in the focus of studies on DL or in new ICT²¹.

One of the greatest advantages of the tool is that it both facilitates the visual presentation of information and improves memorization through the use of an image. According to Afonso et al.²¹:

"The use of this tool in courses that are based on the reflexive critical pedagogical model is extremely important due to the potential for the immediate reorganization of the teaching strategy according to the performance of the participants. It is also worth mentioning the contribution of this analysis to immediate feedback, a fundamental strategy in the evaluation of quality."

Using this tool to evaluate the keywords found in the intervention projects allows assessment of the incorporation of knowledge into the critical-reflective universe of the student. It should also be noted that as these final projects involve a proposed intervention, the words featured are not unconnected to reality, but instead are interlinked with meaningful learning leading to a change in the professional practice of students and consequently in their professional reality.

CONCLUSION

Evaluation is necessary as it promotes a reflexive process of a certain practiced action, with a view to inducing procedural changes, achieving results that correspond more directly to the needs observed by students, the institution and society, building and re-signifying knowledge in a continuous, daily and shared manner²³. Therefore, a Distance Learning approach in Primary Health Care should focus on the organization of the Unified Health System²⁴ itself in the sense of continuous assessment capable of supporting health professionals in reflections on their work process.

Documentary analysis and analysis of the intervention projects of the students of the specialization course in elderly health using Kirkpatrick's levels of evaluation suggested that the experience of the course was successful in terms of variables inherent to the course itself, its content and its didactic-pedagogical approach.

The qualitative analysis of the responses to the semi-structured questionnaires carried out during the course by the students suggests an agile response on the part of the pedagogical team, as well as close interactions with the tutors demonstrated by the degree of satisfaction with the tutoring activities, responding to the responses of the students in an effective way, which is described in literature as fundamental for a positive student respons^16,17,25.

The aspects highlighted by the students in the narratives of their semi-structured questionnaires included content, allowing the inference that not only did tutors present specific training in the area of geriatrics and gerontology, but also maintained a close relationship with the pedagogical team and with teachers of content, as the feedback given to the student was positively evaluated in the narratives.

Although few dropout studies have been aimed at post-graduate courses in Distance Learning, it is noteworthy that even when using more rigid dropout criteria, considering all enrolled students irrespective of whether they entered the virtual platform or not, the specialization course in elderly health had a dropout rate of 38%, lower than expected for distance learning courses aimed at the same public¹⁹. Another key aspect was the high pass rate of around 94%, resulting in 299 intervention projects aimed at the health professional.

There were also indications of changes in the behavior of the graduating students, demonstrated by the analysis of the structured intervention projects and the themes pertinent to the area of action of the students, focused on the health of the elderly.

One of the limitations of the present is that it does not consider the impact of intervention projects developed by the students in their working reality. From this perspective the fourth level of the Kirkpatrick evaluation could be used to highlight the added value to society of a given educational program, and the application of Kirkpatrick's four levels of evaluation, of which the present study employed only the first three, would represent a valid evaluation methodology. However, it should be noted that this analysis must be carried out in an interconnected and systemic manner, where the variables studied in the first level from the impressions of the students in the semi-structured questionnaires can also explain the aspects of adherence and dropout evaluated in the second level, for example. The systemic approach promoted by this evaluation would allow the cause-effect relationship to be exchanged for an analysis where qualitative aspects could influence quantitative evaluations, such as in dropout studies. It also allows that dropout can be explained from the perspective of the recognition of the internal or external factors of the course, which influence its final outcome.

Directing resources towards educational initiatives in areas of fragility can have an impact on the population and the health professionals themselves. The application of Kirkpatrick's fourth level of evaluation to this perspective could identify the added value to society of a given educational program, as it allows the evaluation not only of the application of intervention projects in the practice of health professionals, but also the results of their use in a local context.

REFERENCES

• ¹
  Veras R. Envelhecimento populacional contemporâneo: demandas, desafios e inovações. Rev Saúde Pública [Internet]. 2009 [acesso em 29 jan. 2017];43(3):548-54. Disponível em: http://dx.doi.org/10.1590/S0034-89102009005000025
  » http://dx.doi.org/10.1590/S0034-89102009005000025
• ²
  Brasil. Portaria nº. 2.528, de 19 de outubro de 2006. Aprova a Política Nacional de Saúde da Pessoa Idosa. Diário Oficial da República Federativa do Brasil. 20 out. 2006; Seção 1:142.
• ³
  Hill LH. Interactive influences on health and adult education. News Dir Adult Contin Educ [Internet]. 2016 [acesso em 02 fev. 2018];(149):41-51. Disponível em: http://onlinelibrary.wiley.com/wol1/doi/10.1002/ace.20175/full
  » http://onlinelibrary.wiley.com/wol1/doi/10.1002/ace.20175/full
• ⁴
  Motta LB, Aguiar AC, Caldas CP. Estratégia Saúde da Família e a atenção ao idoso: experiências em três municípios brasileiros. Cad Saúde Pública [Internet]. 2011 [acesso em 06 fev. 2017];27(4):779-86. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102-311X2011000400017&lng=en&nrm=iso
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102-311X2011000400017&lng=en&nrm=iso
• ⁵
  Merhy EE. Em busca do tempo perdido: a micropolítica do trabalho vivo em saúde. In: Merhy EE, Onocko R, organizadores. Agir em saúde: um desafio para o público. São Paulo: Hucitec;1997. p. 71-112.
• ⁶
  Fratucci MVB. Ensino a distância como estratégia de educação permanente em saúde: impacto da capacitação da equipe de estratégia de saúde da família na organização dos serviços [tese]. São Paulo: Universidade de São Paulo, Programa de Pós-Graduação em Odontologia Social; 2015.
• ⁷
  Gomes LF. EAD no Brasil: perspectivas e desafios. Avaliação [Internet]. 2013 [acesso em 03 fev. 2018];18(1):13-22. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1414-40772013000100002&lng=en&nrm=iso
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1414-40772013000100002&lng=en&nrm=iso
• ⁸
  Oliveira VA. O quebra-cabeça da Universidade Aberta do SUS. In: Gusmão CMG. Relatos de uso de tecnologias educacionais na educação permanente de profissionais de saúde no sistema Universidade Aberta do SUS. Pernambuco: Editora Universitária UFPE; 2014. p.11-28.
• ⁹
  Hallett K, Essex C. Evaluating online instruction: adapting a training model to e-learning in higher education. In: 14th Proceedings ED-MEDIA 2002 World Conference on Educational Multimedia, Hypermedia & Telecommunications; 24-29 june 2002; Denver 2002. Norfolk: AACE; 2002.
• ¹⁰
  Comin FS. Avaliação dos processos de ensino-aprendizagemem ações educacionais ofertadas a distância. Temas psicol [Internet]. 2013 [acesso em 04 fev. 2018];21(2):335-46. Disponível em: http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1413-389X2013000200003&lng=pt&nrm=iso
  » http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1413-389X2013000200003&lng=pt&nrm=iso
• ¹¹
  Waddill DD. Action e-learning: an exploratory case study of action learning applied online. Hum Res Dev Int [Internet]. 2006 [acesso em 02 fev. 2018];9(2):157-71. Disponível em: https://www.bvekennis.nl/Bibliotheek/06-0747_Action_E-learning.pdf
  » https://www.bvekennis.nl/Bibliotheek/06-0747_Action_E-learning.pdf
• ¹²
  Martins-Melo FR, Lima MS, Ramos Junior MN, Heukelbach J, Campo MOC. Modalidade de educação a distância na formação profissional em saúde da família: relato de experiência. Rev Bras Med Fam Comunidade [Internet]. 2014 [acesso em 04 fev. 2018];30(9):89-95. Disponível em: https://www.rbmfc.org.br/rbmfc/article/view/486
  » https://www.rbmfc.org.br/rbmfc/article/view/486
• ¹³
  Rodrigues CMC, Ribeiro JLD, Cortimiglia M, Bündchen C. Uma Proposta de instrumento para avaliação da educação a distância. Ensaio Aval Pol Públ Educ [Internet]. 2014 [acesso em 13 fev. 2018];22(83):321-54. Disponível em: http://www.scielo.br/pdf/ensaio/v22n83/a04v22n83.pdf
  » http://www.scielo.br/pdf/ensaio/v22n83/a04v22n83.pdf
• ¹⁴
  Silva LS, Correa EP, Duarte LM, Miranda SA. Formação continuada em educação a distância: percepções sobre as competências na atuação do professor-tutor. EmRede Rev Educ Dist [Internet]. 2016 [acesso em 02 fev 2018];3(2):252-65. Disponível em: https://www.aunirede.org.br/revista/index.php/emrede/article/view/122
  » https://www.aunirede.org.br/revista/index.php/emrede/article/view/122
• ¹⁵
  Fiuza PJ, Sarriera JC. Motivos para adesão e permanência discente na educação superior a distância. Psicol Ciênc Prof [Internet]. 2013 [acesso em 02 fev. 2018];33(4):884-901. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1414-98932013000400009&lng=en&nrm=iso
  » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1414-98932013000400009&lng=en&nrm=iso
• ¹⁶
  Claudino-Kamazaki SG, Capellini VLMF, Oliveira AAS, Pedro KM. Formação de professores em educação especial na modalidade EaD: alguns apontamentos sobre seis cursos de especialização. EaD Foco [Internet]. 2017 [acesso em 27 fev. 2018];7(3):29-41. Disponível em: http://eademfoco.cecierj.edu.br/index.php/Revista/article/view/549/268
  » http://eademfoco.cecierj.edu.br/index.php/Revista/article/view/549/268
• ¹⁷
  Bizarria FPA, Silva MA, Tassigny MM, Carneiro TCJ. Papel do tutor no combate à evasão na EAD: percepções de profissionais de uma instituição de ensino superior. Rev Educ Ciênc Cult [Internet]. 2015 [acesso em 27 fev. 2018];20(1):86-102. Disponível em: https://www.researchgate.net/profile/Maria_Silva161/publication/281601630
  » https://www.researchgate.net/profile/Maria_Silva161/publication/281601630
• ¹⁸
  Portal C. Estratégias para minimizar a evasão e potencializar a permanência em EaD a partir de sistema que utiliza mineração de dados educacionais e learning analytics [Dissertação]. Porto Alegre: Universidade do Vale do Rio dos Sinos, Programa de Pós-Graduação em Educação; 2016
• ¹⁹
  Rinaldo RB. Evasão em curso multiprofissional de especialização em saúde da família: comparação entre modalidade a distância versus presencial [Dissertação]. Universidade de Federal do Rio Grande do Sul, Programa de Pós-Graduação Ensino na Saúde; 2014.
• ²⁰
  Vargas FMA, Trindade MCN, Gouveia GDA, Farias MR. A educação a distância na qualificação de profissionais para o Sistema Único de Saúde: metaestudo. Trab Educ Saúde [Internet]. 2016 [acesso em 04 fev. 2017];14(3):849-70. Disponível em: http://www.scielo.br/scielo.php?pid=S1981-77462016000300849&script=sci_abstract&tlng=pt
  » http://www.scielo.br/scielo.php?pid=S1981-77462016000300849&script=sci_abstract&tlng=pt
• ²¹
  Afonso DH, Monteiro A, Carvalho Júnior PM, Dias CC. Análise qualitativa do portfólio digital na formação pedagógica de preceptores da área de saúde: vantagens do wordle. In: 17º Congresso internacional ABED de educação a distância; 30 agosto - 02 setembro de 2011; Manaus. Manaus: ABED; 2011.
• ²²
  Silva IP, Mercado LPL. Levantamento dos temas TIC e EAD na biblioteca virtual Educ@. Cad Pesqui [Internet]. 2015 [acesso em 02 fev. 2018];45(158):970-88. Disponível em: http://www.scielo.br/scielo.php?pid=S0100-15742015000400970&script=sci_abstract&tlng=pt
  » http://www.scielo.br/scielo.php?pid=S0100-15742015000400970&script=sci_abstract&tlng=pt
• ²³
  Abadi AM, Rehfeldt MJH. Autonomia para aprendizagem: uma relação entre o fracasso e o sucesso dos alunos da Educação a Distância. Práxis Educativa [Internet]. 2016 [acesso em 02 fev. 2018];11(2):310-31. Disponível em: http://www.revistas2.uepg.br/index.php/praxiseducativa/article/viewFile/7107/4772
  » http://www.revistas2.uepg.br/index.php/praxiseducativa/article/viewFile/7107/4772
• ²⁴
  Rangel ML, Barbosa EO, Riccio NCR, Souza JS. Redes de aprendizagem colaborativa: contribuição da Educação a Distância no processo de qualificação de gestores do Sistema Único. Interface Comun Saúde Educ [Internet]. 2012 [aceso em 03 fev. 2018];16(41):545-55. Disponível em: http://www.scielo.br/pdf/icse/v16n41/aop3412.pdf
  » http://www.scielo.br/pdf/icse/v16n41/aop3412.pdf
• ²⁵
  Pedro N, Razera F. A influência do papel do tutor na interação em fóruns de discussão: um estudo em EaD com base na análise de redes sociais. Rev E-Curriculum [Internet]. 2018 [acesso em 27 fev. 2018];16(2):467-93. Disponível em: http://dx.doi.org/10.23925/1809-3876.2018v16i2p467-49
  » http://dx.doi.org/10.23925/1809-3876.2018v16i2p467-49

Publication Dates

• Publication in this collection
  Sep-Oct 2018

History

• Received
  18 Feb 2018
• Reviewed
  06 Aug 2018
• Accepted
  24 Aug 2018