E-learning in Pharmacy Education: what do we know about it?

Abstract

The increase and flexibilization of e-learning in Brazil has generated concern about the low capacity to ensure the quality of institutions and courses. The objective is to identify the literature on experiences of pharmaceutical e-learning. A scope review was carried out, mapping key concepts and evidence. The selected articles, without period restriction, from the Scielo, Pubmed and Scopus databases provided data on the educational level of experience, modality, workload, content delivery method, instruments used, types of topics, evaluation information and purpose of the study. There were 87 articles on preparation for subsequent classes; feedback or continuation of previous classes; to specific training; the comparison between face-to-face and distance education modalities; and distance learning. Among the subjects treated, 51.8% involved pharmaceutical clinic and 42.5% basic science. Positive student assessment was found in more than 92% of cases. Isolation, and deficits in social and analytical skills were negative points. Technology infrastructure can impact learning. In Brazil, the health area has manifested opposition to the use of distance education in undergraduate studies. There is no evidence on the organization of undergraduate distance courses, but there are elements indicating risks to the results of training and development of professional skills in this modality.

Keywords:
Pharmacy education; E-learning; Distance learning

INTRODUCTION

Brazil was surprised in 2017 with Decree n^o. 9.057 (Brasil, 2017a), which has a new regulation that facilitates the offer of distance education in the country. With the publication of this new decree, it is authorized to offer graduation in the e-learning modality, made available totally at distance.

The first important regulatory landmark for distance education in Brazil dates back to the 1990s. Through Law n^o. 9.394/96, e-learning at all levels of education has become a concrete and regulated object (Marques, 2004).

In Brazil, distance education is defined by Decree n^o. 5.622 as:

… educational modality in which the didactic-pedagogical mediation in teaching and learning processes occurs with the use of information and communication media and technologies, with qualified staff, and with access policies, and compatible monitoring and evaluation., The development educational activities by students and professionals of education who are in different places and times … (Brasil, 2005)

Since its publication, the new regulations have generated concern in both professors and students, once it makes possible to accredit higher education institutions for distance learning courses (e-learning) without previous accreditation to offer face-to-face courses in the same area. As ^{DiPiro (2003}) cites, using these technologies, the quality of instruction and the ability of students to main course outcomes should be maintained and evaluated. ^{Alves (2009}) reports that the trajectory of distance education in Brazil is permeated by advances and setbacks, as well as moments of stagnation, caused mainly by the absence of adequate public policies for the sector.

Briefly, the story of the distance learning can be divided into three moments. The initial period, from 1904 to 1923, was marked by the beginning of the offer of courses for people who were looking for jobs, mainly in the sectors of services and commerce. From the 1940s, at the intermediate moment, we had the offer of free courses and educational formative courses, with the emergence of the Brazilian Universal Institute, which used booklets sent via Post Office. In the modern phase, we can mention the creation of the Brazilian Distance Education Association, which has collaborated in the development of distance education in Brazil, promoting the articulation of institutions and professionals (^{Alves, 2009}; ^{Faria, Salvadori, 2010}).

Thus, in the context of public education policies, e-learning emerges as a strategy for expanding enrollment, contributing to overcome the physical and structural limitations of offers traction (^{Arruda, Arruda, 2015}). Also worthy of mention is the creation of the Open University of Brazil (UAB), by Ministry of Education, in 2005, establishing an association of higher education federal institutions in partnership with states and cities, making it possible to internalize training in countrywide through e-learning education (Costa et al., 2015).

Health systems around the world face several challenges in their trajectory, such as demographic changes, the show up of new drugs and the increase in the costs of medical technology, thus generating difficulties in the training of health professionals and in continuing professional development, which are limiting factors for the workforce development. As a result, e-learning has been largely responsive to the growing number of medical education and continuing education programs with e-learning approaches, and has proven to be a flexible, user-centered and easily upgraded tool (^{Ruggeri, Farrington, Brayne, 2013}).

In universities of countries with few traditions in medical education, the largest current investment in the development of the health workforce is related to projects using e-learning. This has been the main modality of offering vacancies to a large contingent of students in regions lacking higher education. Institutions around the world have invested heavily in e-learning technologies, aiming to expand the offer at undergraduate, postgraduate and continuing education (^{Mullan et al., 2012}). E-learning has the potential to provide the dissemination of educational content, in an adaptive way, with training material that can be distributed through electronic media, especially the Internet, which is increasingly ubiquitous in society.

According to ^{Litto (2009}), “Brazil is now receiving its rightful place among countries that make wide use of the e-learning for access to knowledge and certification of skills to the growing population.” Until 2012, e-learning was already part of “...52 Brazilian federal universities operating in 519 poles. In 2010, there were 43.959 undergraduate positions offered in 162 courses” (Costa et al., 2015). However, there is a need for public policies that give better organization and greater credibility. Litto (2009) still ponders: “Distance learning does not have a purpose to compete with a classroom, but to offer a good quality option for those who do not have access to conventional learning”.

The Pharmaceutical Sciences, whose teaching-learning process and the actors of education (school-student-teacher) preserve a previous e-learning tradition, essentially face-to-face, are rapidly becoming part of the e-learning context (Júnior, Batista, 2012). The perspective of developing needs-based pharmacy education dominates the international discussions (^{Anderson et al., 2012}), with several globally agreed pharmacy education statements, including experiential learning and quality assurance standards (International Pharmaceutical Federation - FIP, 2017). The Brazilian National Council of Education has just published the National Curricular Guidelines for Pharmacy Undergraduate Courses (Brasil, 2017b) which emphasize, in addition to training for the pharmaceutical productive sector, health care and management, interaction with health system and society, bringing the challenge of training by active learning methodologies, articulation between theories, practices and continued experiences in scenarios of diversified practices.

In this context, studies involving virtual learning strategies in pharmaceutical education are scarce, which may suggest a still infrequent use of the e-learning modality in this area. In some medical schools, virtual educational tools have been widely disseminated and are considered important resources for teaching (^{Jabbur-Lopes et al., 2012}).

The insertion of e-learning in health education is something challenging, as much as necessary in a country of continental dimensions and as unequal as Brazil (^{Christante et al., 2003}). Is it a strategic alternative in the teaching and training for clinical practice? The purpose of this article is to identify the literature on distance pharmaceutical education experiences, analyzing its purposes, resources, structure and results, constructing a reflection on the consequences and on the impact of the massification of e-learning on pharmaceutical education, considering the country´s political and economic scenarios, as well as, the new regulation to offer undergraduate education entirely at a distance.

MATERIAL AND METHODS

For this study, a scope review was carried out, with the broad aim of mapping the key concepts, the available evidence in this sector of knowledge and the research data, and also examine its extent and identifying gaps in the literature (^{Arksey, O’Malley, 2005}).

All the articles published in the Scielo, PubMed and Scopus databases was selected, without period restriction. The indicators are listed in Table I.

From all the articles found, duplicated studies were excluded. Two subsequent steps were used to separate the studies of interest in the research: a) reading the title of the articles; b) dynamic reading of articles. In addition to searching the databases, a manual search was performed on the references of the selected articles. The entire process was performed in duplicate, and compared. The exclusion criteria applied were: articles relating to other professions further than pharmacy; articles regarding classroom lessons practices; articles on drug research, therapeutic response, professional practices in general.

From the articles, were extracted information regarding: education level, modality, workload, delivery method (synchronous and asynchronous), applied tools, subject types, information about evaluation and purpose of the study.

RESULTS AND DISCUSSION

Characterization of the uses of e-learning

The selection of articles is shown in Figure 1, adding a total of 87 articles that were included in the analysis. We found articles that reported 5 types of uses of distance learning in pharmaceutical education: i) involving preparation for a later class; ii) for feedback or continuation of previous classroom lessons; iii) for specific trainings; iv) analysis and comparison between face-to-face and distance learning modalities; v) distance learning (without detail) (Figure 2).

FIGURE 1
Flowchart of search, exclusion and inclusion of articles.

FIGURE 2
Types of studies found.

TABLE I
Search Strategy

Database	Keywords
PUBMED	(“education, distance”MeSH Terms OR distance learning Text Word OR “Distance Education” or “Distance Learning” OR “Learning, Distance” OR “Correspondence Courses” OR “Correspondence Course” OR “Course, Correspondence”) AND (“education, pharmacy” MeSH Terms OR “Education, Pharmaceutical” OR “Pharmaceutical Education” OR “Pharmacy Education” OR “Education, Pharmaceutic” or “Pharmaceutic Education’’) AND (“Pharmacy OR pharmacist”)
SCOPUS	(“e-learning pharmacy” OR “distance learning” OR “distance education”) AND (“Pharmaceutical Education” OR “Pharmacy Education”) AND (“Pharmacy or pharmacist”)
SCIELO	(“Education, Distance” OR “Educação a Distância”) AND (“Education, Pharmacy” OR “Educação em Farmácia” OR “Education, Pharmacy, Continuing” OR “Educação Continuada em Farmácia”) AND (“Pharmacy” OR “pharmacist”)

• Articles involving the use of social networks, such as Facebook® and Twitter®, were classified as complementary studies. They were not included in the main data analysis.

The articles selected for this review study are listed in Table II.

Most studies using e-learning were developed in undergraduate courses (76.7%) (Figure 3), mostly using distance learning in addition to face-to-face teaching techniques. The two main themes were applied clinics (51.8%), followed by the basic sciences (43.5%). In some studies, the use of e-learning was also observed in remote areas or, far from the academic centers, where the same classes were available at the campus far from the main campus, by streaming or by videoconference (2, 19, 29, 35, 45, 53, 54, 59, 64, 68, 73, 80).

FIGURE 3
Levels of education using e-learning.

TABLE II
Selected articles for review study

N.	Author (year)	Title	Journal
1	Al-Dahir et al. (2014)	Online Virtual-Patient Cases Versus Traditional Problem-Based Learning in Advanced Pharmacy Practice Experiences	Am J Pharm Educ
2	Anderson et al. (2012)	Needs-Based Education in the Context of Globalization	Am J Pharm Educ
3	Aojula et al. (2006)	Computer-based, online summative assessment in undergraduate pharmacy teaching: The Manchester experience	Pharmacy Education
4	Battaglia et al. (2012)	An Online Virtual-Patient Program to Teach Pharmacists and Pharmacy Students How to Provide Diabetes-Specific Medication Therapy Management	Am J Pharm Educ
5	Benedict (2010)	Virtual Patients and Problem-Based Learning in Advanced Therapeutics	Am J Pharm Educ
6	Benedict, Schonder, Mcgee (2013)	Promotion of Self-directed Learning Using Virtual Patient Cases	Am J Pharm Educ
7	Bindoff et al. (2014)	Computer Simulation of Community Pharmacy Practice for Educational Use	Am J Pharm Educ
8	Bourne, Davison (2006)	A self-paced course in pharmaceutical mathematics using web-based databases	Am J Pharm Educ
9	Bracchi et al. (2005)	A distance-learning programme in pharmacovigilance linked to educational credits is associated with improved reporting of suspected adverse drug reactions via the UK yellow card scheme	Br J Clin Pharmacol
10	Brahm, Davis (2010)	Psychiatric pharmacy delivery via an online course: Psychoactive substances	Curr Pharm Teach Learn
11	Brandys et al. (2006)	An e-learning system for pharmacist continuing education in Poland	Pharmacy Education
12	Brock, Smith (2007)	An interdisciplinary online course in health care informatics	Am J Pharm Educ
13	Buxton, De Muth (2013)	Pharmacists’ perceptions of a live continuing education program comparing distance learning versus local learning	Res Social Adm Pharm
14	Buxton et al.(2012)	Professional Development Webinars for Pharmacists	Am J Pharm Educ
15	Buxton (2014)	Pharmacists’ Perception of Synchronous Versus Asynchronous Distance Learning for Continuing Education Programs	Am J Pharm Educ
16	Bykhovsky et al. (2014)	Impact of an Online Educational Program on Pharmacy Students’ Knowledge of Folic Acid and Neural Tube Defects	Journal Of Pharmacy Practice
17	Bynum et al. (2010)	Satisfaction with a Distance Continuing Education Program for Health Professionals	Telem E-Health
18	Carbonaro et al. (2008)	Integration of e-learning technologies in an interprofessional health science course	Medical Teacher
19	Congdon et al. (2009)	Impact of hybrid delivery of education on student academic performance and the student experience	Am J Pharm Educ
20	Crouch (2009)	An Advanced Cardiovascular Pharmacotherapy Course Blending Online and Face-to-Face Instruction	Am J Pharm Educ
21	De Muth, Bruskiewitz (2006)	A Comparison of the Acceptability and Effectiveness of Two Methods of Distance Education: CD-ROM and Audio Teleconferencing	Am J Pharm Educ
22	El-Magboub et al. (2016)	Evaluation of in-class and online discussion meetings in a biopharmaceutics problem-based learning class	Curr Pharm Teach Learn
23	Elliott et al. (2009)	A Pharmacy Preregistration Course Using Online Teaching and Learning Methods	Am J Pharm Educ
24	Erickson et al. (2003)	Lecture versus Web Tutorial for Pharmacy Students’ Learning of MDI Technique.	Ann Pharmacother
25	Faulkner et al. (2005)	Pilot study of a distance-learning methodology used on campus for first professional degree pharmacy students in an integrated therapeutics module	Am J Pharm Educ
26	Fernández-Alemán et al. (2016)	An Empirical Study of Neural Network-Based Audience Response Technology in a Human Anatomy Course for Pharmacy Students	J Med Syst
27	Flowers et al. (2010)	Web-based Multimedia Vignettes in Advanced Community Pharmacy Practice Experiences	Am J Pharm Educ
28	Fox et al. (2011)	Establishing and Maintaining a Satellite Campus Connected by Synchronous Video Conferencing	Am J Pharm Educ
29	Freeman et al. (2006)	Student Perceptions of Online Lectures and WebCTin an Introductory Drug Information Course	Am J Pharm Educ
30	Fuji, Galt (2015)	An Online Health Informatics Elective Course for Doctor of Pharmacy Students	Am J Pharm Educ
31	Garrison et al. (2015)	An asynchronous learning approach for the instructional component of a dual-campus pharmacy resident teaching program	Am J Pharm Educ
32	Gossenheimer et al. (2017)	Impact of distance education on academic performance in a pharmaceutical care course	Plos One
33	Grace et al. (2016)	Preparing health students for interprofessional placements	Nurse Educ Pract
34	Hall et al. (2007)	A Web-based Interprofessional Diabetes Education Course	Am J Pharm Educ
35	Hall et al. (2010)	Application of a technology-based instructional resource in diabetes education at multiple schools of pharmacy: evaluation of student learning and satisfaction	Curr Pharm Teach Learn
36	Hedaya (1998)	Development and evaluation of an interactive internet-based pharmacokinetic teaching module	Am J Pharm Educ
37	Hughes, Schindel (2010)	Evaluation of a professional development course for pharmacists on laboratory values: can practice change?	Int J Pharm Pract
38	Hussein, Kawahara (2006)	Adaptive and Longitudinal Pharmaceutical Care Instruction Using an Interactive Voice Response/Text-to-Speech System	Am J Pharm Educ
39	Isaacs et al. (2017)	Student-generated e-learning for clinical education	Clin Teach
40	Jabbur-Lopes et al. (2012)	Virtual Patients in Pharmacy Education	Am J Pharm Educ
41	Kaplan et al. (1996)	Adaptation of Different Computerized Methods of Distance Learning to an External PharmD Degree Program	Am J Pharm Educ
42	Kennedy et al. (2003)	Distance education: Using compressed interactive video technology for an entry-level Doctor of Pharmacy program	Am J Pharm Educ
43	Kidd, Stamatakis (2006)	Comparison of students’ performance in and satisfaction with a clinical pharmacokinetics course delivered live and by interactive videoconferencing	Am J Pharm Educ
44	Lancaster et al. (2011)	Online lecture delivery paired with in class problem-based learning … Does it enhance student learning?	Curr Pharm Teach Learn
45	Leikola et al. (2009)	Continuing Education Course to Attain Collaborative Comprehensive Medication Review Competencies	Am J Pharm Educ
46	Leong et al. (2015)	Student perspectives of an online module for teaching physical assessment skills for dentistry, dental hygiene, and pharmacy students	J Interprof Care
47	Lewis, Sewell (2007)	Providing Formative Feedback From a Summative Computer-aided Assessment	Am J Pharm Educ
48	Limpach et al. (2008)	Effectiveness of human anatomy education for pharmacy students via the internet	Am J Pharm Educ
49	Loke et al. (2011)	How pharmacy students made meaning of a clinical case differently in paper- and simulation-based workshops	Br. J. Educ. Technol
50	Lust (2014)	An Online Course in Veterinary Therapeutics for Pharmacy Students	Am J Pharm Educ
51	Malone et al. (2004)	The Development and Structure of a Web-based Entry-level Doctor of Pharmacy Pathway at Creighton University Medical Center	Am J Pharm Educ
52	Maphanta, Johnson (1999)	Use of conferencing software for the distance education of renal pharmacotherapy	Am J Pharm Educ
53	McLaughlin et al. (2004)	Impact of distance learning using videoconferencing technology on student performance	Am J Pharm Educ
54	McLaughlin et al. (2013)	Pharmacy Student Engagement, Performance, and Perception in a Flipped Satellite Classroom	Am J Pharm Educ
55	Mehvar (2010)	A Participation Requirement to Engage Students in a Pharmacokinetics Course Synchronously Taught at a Local and Distant Campus	Am J Pharm Educ
56	Mobley (2002)	Adaptation of a hypertext pharmaceutics course for videoconference-based distance education	Am J Pharm Educ
57	Moridani (2007)	Asynchronous video streaming vs. synchronous videoconferencing for teaching a pharmacogenetic pharmacotherapy course	Am J Pharm Educ
58	Mullan et al. (2012)	The Medical Education Partnership Initiative: PEPFAR’s effort to boost health worker education to strengthen health systems	Health Aff Proj Hope
59	Nesterowicz et al. (2014)	Validating e-learning in continuing pharmacy education: User acceptance and knowledge change	BMC Med Educ
60	Nesterowicz et al. (2015)	e-learning in continuing pharmacy education is effective and just as accepted as on-site learning	Pharmacy Education
61	Nuffer, Duke (2013)	A Comparison of Live Classroom Instruction and Internet-Based Lessons for a Preparatory Training Course Delivered to 4th Year Pharmacy Students	J Sci Educ Technol
62	Obreli-Neto et al. (2016)	Evaluation of the effectiveness of an Internet-based continuing education program on pharmacy-based minor ailment management: a randomized controlled clinical trial	Braz. J. Pharm. Sci
63	Poirier, O’Neil (2000)	Use of web technology and active learning strategies in a quality assessment methods course	Am J Pharm Educ
64	Porter et al. (2014)	Comparison of online versus classroom delivery of an immunization elective course	Am J Pharm Educ
65	Ried, Byers (2009)	Comparison of Two Lecture Delivery Platforms in a Hybrid Distance Education Program	Am J Pharm Educ
66	Ried, McKenzie (2004)	A Preliminary Report on the Academic Performance of Pharmacy Students in a Distance Education Program	Am J Pharm Educ
67	Ried (2010)	A distance education course in statistics.	Am J Pharm Educ
68	Robertson, Shrewsbury (2011)	Video teleconferencing in the compounding laboratory component of a dual-campus doctor of pharmacy program.	Am J Pharm Educ
69	Rochester, Pradel (2008)	Students’ Perceptions and Satisfaction With a Web-Based Human Nutrition Course	Am J Pharm Educ
70	Ruehter et al. (2012)	Use of Online Modules to Enhance Knowledge and Skills Application During an Introductory Pharmacy Practice Experience	Am J Pharm Educ
71	Salter et al. (2014)	Long-term Effectiveness of Online Anaphylaxis Education for Pharmacists	Am J Pharm Educ
72	Sancho et al. (2006)	A Blended Learning Experience for Teaching Microbiology	Am J Pharm Educ
73	Savela, Enlund (1996)	Public Radio as a Means of Continuing Education in Pharmacy	Am J Pharm Educ
74	Sibbald (2011)	Elective Self-Care Course Emphasizing Critical Reasoning Principles	Am J Pharm Educ
75	Steinberg, Morin (2011)	Academic Performance in a Pharmacotherapeutics Course Sequence Taught Synchronously on Two Campuses Using Distance Education Technology	Am J Pharm Educ
76	Villaume et al. (2006)	Learning Motivational Interviewing: Scripting a Virtual Patient	Am J Pharm Educ
77	Vovides et al. (2014)	A Systems Approach to Implementation of eLearning in Medical Education	Acad Med
78	Wade et al. (1999)	Assessment of student performance in an advanced pharmacokinetics course taught by three methods of instructional delivery	Am J Pharm Educ
79	Ward et al. (2006)	Does instructor presence in the classroom influence examination scores in a therapeutics course delivered via interactive video-conferencing technology?	Pharm Educ
80	Warren et al. (2015)	Online induction programme with webinar: a case study	J Appl Res High Educ
81	West et al. (2004)	Developing a diabetes review course for rural health professionals using videoconferencing.	Diabetes Educ
82	Willett, Bouldin (2004)	Development and Assessment of an Online Elective Toxicology Course	Am J Pharm Educ
83	Woodruff et al. (2014)	Advanced Screencasting With Embedded Assessments in Pathophysiology and Therapeutics Course Modules	Am J Pharm Educ
84	Zary et al. (2006)	Development, implementation and pilot evaluation of a Web-based Virtual Patient Case Simulation environment - Web-SP	BMC Med Educ
85	Zlotos et al. (2010)	A Web-based Tool for Teaching Pharmacy Practice Competency	Am J Pharm Educ
86	Zlotos et al. (2015)	Integration of an Online Simulated Prescription Analysis into Undergraduate Pharmacy Teaching Using Supplemental and Replacement Models	Am J Pharm Educ
87	Zlotos et al. (2016)	A Scenario-Based Virtual Patient Program to Support Substance Misuse Education	Am J Pharm Educ

The purpose for the use of e-learning was, in 33.3% of the studies, as a preparation tool for the next face-to-face lesson (1, 4, 5, 6, 7, 8, 20, 21, 29, 33, 35, 44, 50, 51, 55, 61, 63, 64, 66, 67, 70, 72, 75, 82, 83, 84, 86). These articles highlight several ways to solve how e-learning can be used in the distribution of learning objects: texts for previous reading to the classroom (available in the virtual environment - moodle), exercises involving clinical cases, virtual patients, videos and presentations with the content to be discussed in classroom, for example.

Educators have described internet-based education as an opportunity to enhance student learning (35). Virtual patients and problem-based learning are strategies found in several studies (1, 4, 5, 38, 76, 84, 85, 87) as a complementary pedagogical tool used in e-learning in order to stimulate self-management of the educational process and co-responsibility for the student’s learning (20, 27, 72, 84). As concludes ^{Sancho et al. (2006}), the development of competencies can be achieved through the combination of different learning scenarios. ^{Fernández-Alemán et al. (2016}) argues that about 87% of US pharmacy colleges use such tools as an active form of complementary learning.

The growing technological advance allows the easy distribution of learning objects by tools that help the student developing their autonomy, critical sense and professional skills (38, 53). ^{Benedict, Schonder and Mcgee (2013}), justifies this practice by saying “in Problem Based Learning sessions, students are active participants as they recognize learning deficits, seek new knowledge, apply new knowledge across varying landscapes, formulate plans, and determine the plan’s parameters of success or failure”. This perspective, therefore, runs through the face-to-face training or the e-learning courses.

Other purposes of using e-learning were feedback of previous face-to-face lessons, training and capacity-building for specific activities, studies comparing methods of content delivery and distance learning on campuses far from each other.

Most of the selected articles were published in the United States (67.8%) (Figure 4), showing that other countries, despite using e-learning in pharmaceutical education, haven’t published so much on the subject in the journals covered by the databases searched. One American magazine on ​​pharmaceutical education concentrates 80% of published articles, which may influence the research and publication of this theme.

FIGURE 4
Countries of publication of articles.

In most of the studies, the report of more than one electronic distributed tool was observed. Online texts (including supporting material), clinical cases and pre and post activity tests were the most used instruments, almost always associated with other forms of delivery, such as forums and debates (synchronous and asynchronous). Techniques using videos were also used, almost always for distribution of theoretical content. In addition, videoconferences, webnairs, slideshows, apps and audio were found and classified as “Others”, as shown in Table III.

TABLE III
Used tools

Used tools	Percentage of use
Texts	54%
Forums and discussions	52,9%
Quizzes	36,8%
Videos	54%
Others	54%

In relation to tutoring and evaluation, only 14.3% of the courses did not use tutoring in the activity (7, 9, 13, 36, 46, 71, 72, 73, 87), and only 11% did not require assessment in order to validate and/or certify student approval (13, 16, 18, 33, 39, 49, 76, 87).

It was noticed that many studies using e-learning methods compared synchronous and asynchronous techniques, as well as the electronic and face-to-face distribution of learning objects (13, 15, 19, 20, 21, 43, 44, 49, 57, 61, 65). In this analysis, it was observed that 60.5% of the studies reported the use of asynchronous transmission distribution. This type of communication is intended to make e-learning a flexible approach to meeting the students´ needs. Its importance is evidenced by ^{Moridani (2007}), who argues that “the students indicated that the course was well organized and they enjoyed the flexibility that video streaming offered” and by ^{Buxton (2014}) that “distance learning offers the independence of location and asynchronous learning adds the flexibility of time”. ^{Porter, Pitterle and Hayney (2014}) reports that “students in the online group felt that they were flexible enough to complete the course at their own pace (88%) and were able to pause or watch a lecture again (45%), what were advantages to this delivery method”.

Many studies point the necessity to develop and improve the implementation of highly efficient technological tools to guarantee the quality and consistency of the teaching and learning process with the expected flexibility and autonomy (27, 70, 84).

Flexibility, however, presented itself as a barrier to some students, reducing their involvement with the discussion, since the participants who responded to the forums before had to wait longer until all had completed the task, creating a very large space of time between postings and discussions (31).

Students’ perception of e-learning uses

Most assessments used satisfaction scales, Likert’s, or Bloom’s Taxonomy to analyze students’ perceptions. From the 87 eligible articles for analysis, 55 cited students’ perceptions or feedback (1, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 17, 19, 20, 21, 22, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 38, 39, 41, 43, 45, 46, 48, 52, 53, 54, 57, 59, 60, 61, 62, 64, 67, 69, 70, 72, 73, 74, 76, 81, 82, 83, 84, 85,86). In 92.7% of the studies, the students’ evaluation was positive regarding the use of technologies as a way to mediate teaching, and also in relation to the tool used, leading to the majority of students believed that they saved time by takin the online courses (82). In addition, they believe that the use of e-learning enabled an improvement in the learning process and the use of the subject, besides showing the gaps in students’ learning (6, 8, 26, 29, 40, 46, 79, 83). ^{Hall et al. (2010}), indicates that most students agreed that the use of technology-based teaching challenged them intellectually.

From the point of view of some students, flexibility is a positive factor for this teaching methodology, highlighting “the ability to set my own pace” and “independent learning” as potentialities of e-learning (8, 10, 31, 32, 39, 53, 69, 71, 82). As ^{Gossenheimer et al. (2017}) cites, “the point most highlighted as an advantage of the Internet was that of convenience, allowing students to study without leaving home”. In addition, the majority of students (78%) indicated that replacing lecture with virtual patients allowed for better use of faculty contact time with them (6).

In the study of ^{Porter, Pitterle and Hayney (2014}), students who experienced the classroom delivery of the course felt that method did not allow for flexibility with their schedules while students in the online group felt that method did.

On the other hand, a portion of the students prefer the format of face-to-face seminars because of greater interaction among participants, considering that distance learning can cause professional isolation and training without interprofessional relationship, since there is no face-to-face interaction between the student-student and student-professor (31, 61, 75). In this logic, because they do not develop social skills, this process contributes to, besides a difficulty in solving problems and questions, a deficiency in the analytical sense of professionals and the low adherence to student movements, for example (31, 32).

Even with a general positive trend of evaluation, the use of e-learning in pharmacy is not always good evaluated. It’s possible to find feedback from students who classify the classroom teaching method as being better (29, 31, 75), not adapting to content teaching with various techniques, preferring to receive information using only one learning style, including when asked to choose among visual (learning from graphs, charts, flow diagrams), auditory (learning from speech), printed word (learning from reading and writing), or kinesthetic (touch, hearing, taste, and sight) (65, 86). In some studies, such as that of ^{Moridani (2007}), more than two-thirds of students´ feedback has gained negative perceptions about the online learning experience.

^{Ward, Garrett, Marsh (2006}) concludes that about 77% of students believe that their performance is better when the material is delivered face-to-face. In this study, only one student reported better performance in online learning (79). According to ^{Hussein and Kawahara (2006}), in their study, less than half of the students agree that the online course has helped them gain additional confidence in their ability to apply therapeutic information, besides scoring the use of the system as difficult and cumbersome for teaching.

Moreover, when asked to respond to the statement that all courses except for laboratories should be delivered online, most of the students (73%) of ^{Porter, Pitterle and Hayney (2014}) study disagreed.

Structure and planning of e-learning courses

The implementation of technological tools for education requires a long planning (84), the existence of infrastructure and high initial financing and investments (41), and may present other obstacles even after its implementation (33, 64). The commitment of the university and professors to invest in time to design, development, and implementation such teaching strategies is extensive (4, 6, 65, 79). The creation of materials such as clinical cases, presentations and videos requires additional time, requiring in some cases 15 to 22 hours of time for professors and tutors (1, 74, 84). Besides, as ^{Ried and Byers (2009}) mention, the decision to choose the platform is highly complex, since not all delivery platforms cost the same to develop and deliver.

In addition, it’s necessary to constantly evaluate and maintain these services, according to feedback from students and tutors. Negative impressions have been related to technological problems in materials produced and used: “Negative impressions of the online learning environment may also have been related to technological issues (eg, poor audio recording …)” (31). Technical difficulties were also reported by students and tutors with a use of software (7, 37, 72, 83), evidencing the need for training for those involved (28, 48). These reported hurdles are of great importance, as they can impact on learning, as cited by ^{Hall et al. (2010}): “Regarding technical aspects of the course, 54% felt some difficulty with the program that prevented their learning”.

^{Benedict, Schonder and Mcgee (2013}), using the proposed block of obstacles faced in structuring e-learning courses, groups the difficulties encountered in three blocks: cultural, procedural and academic. The first block concerns the institution’s attitudes towards technology as a pedagogical tool, in other words, how the institution sees distance learning in the training for students. This block is also related to the subjectivity of the subjects in the process of formation, since not all students learn so effectively through the same teaching methodology.

In the cultural block, there are relations with the level of digital inclusion. Considering the students age, whether or not they belong to this new generation more accustomed to the use of technology in their daily lives, one can expect that some students will have more or less affinity with the technology used to mediate the teaching, for example, checking their e-mails daily and being able to access the home platform with the knowledge to complete the set of tasks (32, 41). Therefore, the use of multiple teaching methods in a course is necessary to ensure that all learners are supported (6).

Although any obstacle is capable of destabilizing the implementation of technologies in education, ^{Benedict, Schonder and Mcgee (2013}) considers the procedural block to be more challenging. As this block addresses the evaluation and control of workflows, the proper implementation of the chosen technologies as well as their correct functioning depend on adequate planning. That is, the implementation and evaluation processes depend on how the instructor and the institution intend to deliver the course. For example, in the case of delivery entirely through the internet, factors that may have an impact on learning, such as server capacity, connection speed, information security, choice of a synchronous or asynchronous method, and maintenance (67).

Academic obstacles deal with the degree to which a given technological tool meets the pedagogical objectives expected by the university. In other words, they will reflect the extent to which technology enhances the educator’s ability to engage students.

The e-learning use evaluation, whether in the form of tools, disciplines or undergraduate and postgraduate courses, in most studies is based only on feedback from students’ perceptions, or scores on pre- and post-use tests of the e-learning tool (1, 8, 10, 11, 12, 15, 19, 21, 25, 29, 31, 33, 34, 37, 39, 41, 43, 45, 48, 52, 53, 59, 60, 61, 62, 63, 64, 67, 68, 69, 73, 82, 84). There is a gap in the studies involving the quality of the long-term e-learning, according to ^{Salter et al. (2014}). The results found in their study show that the use of e-learning was promising in tests carried out 7 months after the use of the technological instrument. On the other hand, ^{Zlotos et al. (2016}) cites in his study that “There was a significant increase in knowledge score immediately after program use, with some retention six months later”. Thus, evaluating the use of such teaching methods may be imprecise, since there are few studies evaluating effectively its final product, in other words, the application of long-term learning.

Finally, the use of technological tools requires the availability of electronic devices. Thus, students must own or rent laptops (48, 52, 57) as well as a high-speed data connection to minimize any losses during sessions (69), which can happen when tasks are performed on home networks. In addition, some methods make it impossible to use mass education when using telephone lines, for example, generating the need for student waiting (38).

Comparison between e-learning and face-to-face education strategies

The results of the present study were based on a comparison between the methods used for classroom delivery, e-learning and blended methods (12, 18, 19, 20, 29, 34, 37, 42, 45, 53, 54, 64, 68, 72, 73, 79, 80). In general, it was not possible to observe significant statistical analyzes between teaching methods (7, 19, 25, 54, 75). The blend approach enables course coordinators to take advantage of online activities unique to that course, while having face-to-face discussions with students (20).

Some studies show an equal or greater performance in distance learning when compared to the traditional classroom method (4, 10, 19, 20, 26, 31, 32, 39, 44, 48, 69, 71, 72, 82). Studies have argued that this performance may be higher in e-learning than in the fact students need to be more aware of the need for individual study, self-questioning, and commitment to the proposed activities (32, 39).

In these cases, students have more support material when compared to traditional teaching and, moreover, distance learning provides a greater autonomy to manage the study time, in order to reconcile with their work or financial condition (10, 20, 31, 32, 69, 82). Unlike the traditional “passive” classroom where the student often only listens, in distance learning it will be necessary to reason and reflect on each situation found, such as in clinical case forums and simulation of virtual patients, for example (38).

Some articles even suggest that, despite positive feedbacks, students still prefer traditional teaching as the primary form of learning (61). As cite ^{Rochester and Pradel (2008}), few students agree that forms of online content delivery should replace traditional delivery. It is also possible to find evidence that students disagree with the substitution of e-learning classes, even in non-laboratory courses (64). In addition to the students, some professors suggest that the use of e-learning is associated with the traditional method, and still agree that the face-to-face class is more important (61). In ^{Sancho et al. (2006}) study, most students agree that “the instruction based on virtual systems to be an essential complement to traditional teaching methods”. Also, as citation ^{Porter, Pitterle and Hayney (2014}), “The majority of students in both groups preferred taking the course in the classroom or a blended setting”.

Considerations for reflection about e-learning use in Brazil

The current great interest in the use of e-learning for the training of health professionals has found justification in the literature and public politics as a way to broaden the reach of education to regions and populations that need to expand and qualify the health workforce (^{Michelo et al., 2017}; ^{Mullan et al., 2012}). In Brazil, access to higher education is still a national challenge, in this way, the demand for distance courses by the students has grown considerably and new educational institutions seek to adhere to this modality (^{Gossenheimer, Carneiro, Castro, 2017}). We have a historical deficit of inclusion of young people in the university: Higher education attendance in Brazil was 1.5% during the 1960’s, considering the gross rate of people aged between 18 and 24, thus, distance learning has become an interesting alternative to complement face-to-face learning (Gossenheimer, Carneiro, Castro, 2017). In recent years the attendance is greater, but is still very low if compared with countries such as some European ones (more than 50%) or Chile (58%) (^{Amaral, 2016}). According to the Census of Higher Education (INEP/MEC, 2013) the percentage of people attending higher education represents almost 28.2% of the population aged between 18-24 and around 14.6% are in the age theoretically recognized the ideal to study at this level of education. The current National Plan of Education establishes the goal of 50% as the gross rate in 2025.

The graduation of pharmacists in Brazil was strongly influenced by liberal policies, such as Law nº. 9394/1996 and since then the growth in the number of pharmacy courses has been impressive. At each moment new courses are registered, and in 2017 there were more than 600, distributed all over the regions and in the interior, despite maintaining concentration in the large centers (International Pharmaceutical Federation - FIP, 2017). With more than 200.000 registered pharmacists, Brazil has already reached an average of 1 pharmacist per 1,000 inhabitants, the same found in European countries and the United States (International Pharmaceutical Federation - FIP, 2017). The growth in the number of vacancies in pharmacy schools in Brazil should be linked to the social need of this professional. Pharmaceutical services have grown in importance and volume across all sectors and may result in even greater demand by professionals as professional skills actually meet social and health service needs. This demand, however, is not properly estimated and accounted for, there is no planning of the pharmaceutical workforce for the country (International Pharmaceutical Federation - FIP, 2017). Therefore, the opening amount of e-learning vacancy in Brazil does not show consistency with the international criteria normally employed to define the need for the workforce.

The health area in Brazil has manifested itself in opposition to the e-learning as a modality for offering undergraduate courses, as manifested by Resolution n^o. 515/2016 of the National Health Council (Brasil, 2016), which is the highest instance of popular participation in the constitutionally guaranteed health system. The resolution points out the special concern of the area with regard to the subjects of assistance and practices that address the care/attention in individual and collective health. Additionally, in Pharmacy, training should still cover skills related to the development and production of medicines and clinical-laboratory analyzes, which require intense training of professional practice.

^{Ruggeri, Farrington and Brayne (2013}) suggest that the positive results in the use of e-learning in the health area are directly related to their organization, structure and commitment of the professors. The studies also point the importance of the degree of interest and acceptance on the part of the students. The autonomy of the student, necessary for the process of distance learning places him, in the protagonism of the process. This emphasizes the need for high quality basic education capable of forming citizens capable of taking responsibility for the learning process. This is a particularly important issue for the Brazilian case: in international comparison, the performance of Brazilian students of 15 years in science, reading and mathematics, is considered very low (^{OECD, 2017}). In such scenario, considering fully distance education as an appropriate remedy to address higher education gaps is an announced risk.

Regardless of the method chosen for delivering the content (asynchronous video, live lecture or synchronous videoconference), it’s clearthat there must be a harmony between the three main types of learning: the cognitive, affective and psychomotor, which are found - or are expected to be - in traditional teaching (Clark, 1999; ^{McLaughlin et al., 2004}). The cognitive domain is related to the development of knowledge, understanding, application, analysis of results, synthesis and evaluation. The affective domain refers to students’ subjective emotions, such as feelings, motivations, enthusiasm, ethics, and attitudes. The psychomotor domain involves the development of the practical skills involved in learning (^{Moridani, 2007}). Experimentation, living with practice with its reflection, resignification, relationship with theory and return to the real context in vocational training are key and most critical issues in the scenario of the training of pharmacists prepared to meet the demands of the future (^{Loke et al., 2011}). The use of tools and technologies for e-learning cannot replace or diminish the importance of this movement in learning. In the context of pharmacy training in Brazil, the new National Curriculum Guidelines published in 2017 (Brasil, 2017b) reintroduced the need for practice scenarios from the initial periods in academic laboratories and health services to develop a broad set of general and specific skills and competences.

A major barrier to understanding the impact of health worker training using the Internet is the limited scope of publications and evaluation studies. For the most part, the scope of evaluation is usually limited to user pleasure and satisfaction rather than to the quality of learning and effectiveness of the applied method. In this context, as ^{Gossenheimer, Carneiro and Castro (2017}) points out, it is still necessary to perform a review to measure the quality of the studies, making it possible to confirm how much the performance may be related to the effectiveness of the course. Thus, this evaluation is not sufficient to suggest the implementation of such teaching programs (^{Ruggeri, Farrington, Brayne, 2013}).

CONCLUSION

This study allowed a view of the current scenario of e-learning in Pharmacy higher education, showing that its use has been growing in the form of a collaborative, rather than a substitute, teaching tool. The studies found focus on describing the analysis of experiences of use of e-learning as a complementary tool for pharmaceutical education at the undergraduate, postgraduate and training levels.

The benefits of using e-learning in the medical field and the programs in use should be evaluated within their specific contexts. In addition to its social need for training and intrinsic capacity to expand training, the use of e-learning needs to be evaluated in terms of investments in infrastructure of the University, students and the country, such as the provision of quality internet and low cost in the most remote regions.

No studies and insufficient evidence have been found on the organization of a pharmacy graduation entirely by e-learning. We found some specific studies analyzing in a “superficial” way some strategies of the use of e-learning as a pedagogical complement tool or, delivery of educational content, not configuring in the literature support for the broad and unrestricted application of e-learning as the main pedagogical strategy.

References

• ¹
  Alves JRM. A história da EAD no Brasil. In: Litto FM, Formiga M, editor. Educação a Distância O Estado Arte, vol. 1, São Paulo: Pearson Education Brasil; 2009, p.9-13.
• ²
  Amaral NC. A educação superior brasileira: dilemas, desafios e comparações com os países da OCDE e do BRICS. Rev Bras Educ. 2016,21(66):717-36. doi:10.1590/S1413-24782016216637.
  » https://doi.org/10.1590/S1413-24782016216637
• ³
  Anderson C, Bates I, Brock T, Brown AN, Bruno A, Futter B, et al. Needs-based education in the context of globalization. Am J Pharm Educ. 2012,76(4):56. doi:10.5688/ajpe76456.
  » https://doi.org/10.5688/ajpe76456
• ⁴
  Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005,8(1):19-32. doi:10.1080/1364557032000119616.
  » https://doi.org/10.1080/1364557032000119616
• ⁵
  Arruda EP, Arruda DEP. Educação à distância no Brasil: políticas públicas e democratização do acesso ao ensino superior. Educ Em Rev. 2015,31(3):321-38. doi:10.1590/0102-4698117010.
  » https://doi.org/10.1590/0102-4698117010
• ⁶
  Benedict N, Schonder K, McGee J. Promotion of self-directed learning using virtual patient cases. Am J Pharm Educ. 2013,77(7):151. doi:10.5688/ajpe777151.
  » https://doi.org/10.5688/ajpe777151
• ⁷
  Brasil. Decreto nº. 5.622, de 19 de dezembro de 2005. Regulamenta o artigo 80 da Lei no 9.394, de 20 de dezembro de 1996, que estabelece as diretrizes e bases da educação nacional. Dia´rio Oficial da Unia~o 20 dez. 2005; Seção 1.
• ⁸
  Brasil. Ministério da Saúde. Conselho Nacional de Saúde. Resolução no. 515, de 07 de outubro de 2016. Diário Oficial da União 11 nov 2016; Seção 1.
• ⁹
  Brasil. Decreto nº. 9.057, de 25 de maio de 2017. Regulamenta o artigo 80 da Lei no 9.394, de 20 de dezembro de 1996, que estabelece as diretrizes e as bases da educação nacional. Diário Oficial da União 26 mai 2017a; Seção 1.
• ¹⁰
  Brasil. Ministério da Educação. Resolução CNE/CES no. 6 de 19 de outubro de 2017. Institui as Diretrizes Curriculares Nacionais do Curso de Graduação em Farmácia e dá outras providências. Diário Oficial da União 20 out 2017b; Seção 1.
• ¹¹
  Buxton EC. Pharmacists' perception of synchronous versus asynchronous distance learning for continuing education programs. Am J Pharm Educ. 2014,78(1):8. doi:10.5688/ajpe7818.
  » https://doi.org/10.5688/ajpe7818
• ¹²
  Christante L, Ramos MP, Bessa R, Sigulem D. O papel do ensino a distância na educação médica continuada: uma análise crítica. Rev Assoc Médica Bras. 2003,49(3):326-9. doi:10.1590/S0104-42302003000300039.
  » https://doi.org/10.1590/S0104-42302003000300039
• ¹³
  Clark D. Bloom's taxonomy of learning domains, 1999. [citad 2018 Feb 23]. Available from: http://www.nwlink.com/~donclark/hrd/bloom.html
  » http://www.nwlink.com/~donclark/hrd/bloom.html
• ¹⁴
  Costa ME de O, Santos MS, Barbosa AL da R. Educação a distância e as bibliotecas universitárias: uma interação necessária. Perspect Em Ciênc Informação. 2015,20(2):38-57.
• ¹⁵
  DiPiro JT. Is the quality of pharmacy education keeping up with pharmacy school expansion? Am J Pharm Educ. 2003,67(2):48. doi:10.5688/aj670248.
  » https://doi.org/10.5688/aj670248
• ¹⁶
  Faria AA, Salvadori A. A educação a distância e seu movimento histórico no Brasil. Rev Fac St Cruz. 2010,8(1):15-22.
• ¹⁷
  Fernández-Alemán JL, López-González L, González-Sequeros O, Jayne C, López-Jiménez JJ, Carrillo-de-Gea JM, et al. An empirical study of neural network-based audience response technology in a human anatomy course for pharmacy students. J Med Syst. 2016,40(4):85. doi:10.1007/s10916-016-0440-6.
  » https://doi.org/10.1007/s10916-016-0440-6
• ¹⁸
  Gossenheimer AN, Bem T, Carneiro MLF, de Castro MS. Impact of distance education on academic performance in a pharmaceutical care course. Plos One. 2017, 12(4):e0175117. doi:10.1371/journal.pone.0175117.
  » https://doi.org/10.1371/journal.pone.0175117
• ¹⁹
  Gossenheimer AN, Carneiro MLF, Castro MS de. estudos comparativos entre educação a distância e presencial em cursos da área da saúde: Uma revisão. EmRede - Rev Educ Distância. 2017,4(1):73-90.
• ²⁰
  Hall DL, Corman SL, Drab SR, Smith RB, Meyer SM. Application of a technology-based instructional resource in diabetes education at multiple schools of pharmacy: evaluation of student learning and satisfaction. Curr Pharm Teach Learn. 2010,2(2):108-13. doi:10.1016/j.cptl.2010.01.007.
  » https://doi.org/10.1016/j.cptl.2010.01.007
• ²¹
  Hussein G, Kawahara N. Adaptive and longitudinal pharmaceutical care instruction using an interactive voice response/text-to-speech system. Am J Pharm Educ. 2006,70(2):37. doi:10.5688/aj700237.
  » https://doi.org/10.5688/aj700237
• ²²
  INEP/MEC. Censo da educação superior 2013. [citad 2018 Feb 23].Available from: http://download.inep.gov.br/educacao_superior/censo_superior/apresentacao/2014/coletiva_censo_superior_2013.pdf
  » http://download.inep.gov.br/educacao_superior/censo_superior/apresentacao/2014/coletiva_censo_superior_2013.pdf
• ²³
  International Pharmaceutical Federation (FIP). Research, development and evaluation strategies for pharmaceutical education and the workforce: A global report. The Hague: International Pharmaceutical Federation; 2017.
• ²⁴
  Jabbur-Lopes MO, Mesquita AR, Silva LMA, De Almeida Neto A, Lyra DP. Virtual patients in pharmacy education. Am J Pharm Educ. 2012,76(5):92. doi:10.5688/ajpe76592.
  » https://doi.org/10.5688/ajpe76592
• ²⁵
  Júnior A de FS, Batista H de S. Opinião de estudantes universitários sobre a educação a distância (EaD), no contexto das ciências farmacêuticas. ETD - Educ Temática Digit. 2012,14(2):17.
• ²⁶
  Litto F. O retrato frente/verso da aprendizagem a distância no Brasil 2009. ETD - Educ Temática Digit. 2009,10(2):108-22.
• ²⁷
  Loke S-K, Tordoff J, Winikoff M, McDonald J, Vlugter P, Duffull S. SimPharm: How pharmacy students made meaning of a clinical case differently in paper- and simulation-based workshops: SimPharm: Differences in meaning-making. Br J Educ Technol. 2011,42(5):865-74. doi:10.1111/j.1467-8535.2010.01113.x.
  » https://doi.org/10.1111/j.1467-8535.2010.01113
• ²⁸
  Marques C. Ensino a distância começou com cartas a agricultores, 2004. [citad 2017 Dec 29]. Available from: http://www1.folha.uol.com.br/folha/educacao/ult305u396511.shtml
  » http://www1.folha.uol.com.br/folha/educacao/ult305u396511.shtml
• ²⁹
  McLaughlin JE, Supernaw RB, Howard KA. Impact of distance learning using videoconferencing technology on student performance. Am J Pharm Educ. 2004,68(3).
• ³⁰
  Michelo C, Zulu J, Simuyemba M, Andrews B, Katubulushi M, Chi B, et al. Strengthening and expanding the capacity of health worker education in Zambia. Pan Afr Med J. 2017,27. doi:10.11604/pamj.2017.27.92.6860.
  » https://doi.org/10.11604/pamj.2017.27.92.6860
• ³¹
  Moridani M. Asynchronous Video streaming vs. synchronous videoconferencing for teaching a pharmacogenetic pharmacotherapy course. Am J Pharm Educ. 2007,71(1):16. doi:10.5688/aj710116.
  » https://doi.org/10.5688/aj710116
• ³²
  Mullan F, Frehywot S, Omaswa F, Sewankambo N, Talib Z, Chen C, et al. The Medical Education Partnership Initiative: PEPFAR's effort to boost health worker education to strengthen health systems. Health Aff Proj Hope. 2012,31(7):1561-72. doi:10.1377/hlthaff.2012.0219.
  » https://doi.org/10.1377/hlthaff.2012.0219
• ³³
  OECD. PISA: Programme for International Student Assessment, 2017. [citad 2018 Jan 10]. Available from: http://www.compareyourcountry.org/pisa?lg=en
  » http://www.compareyourcountry.org/pisa?lg=en
• ³⁴
  Porter AL, Pitterle ME, Hayney MS. Comparison of online versus classroom delivery of an immunization elective course. Am J Pharm Educ. 2014,78(5):96. doi:10.5688/ajpe78596.
  » https://doi.org/10.5688/ajpe78596
• ³⁵
  Ried LD, Byers K. Comparison of Two Lecture Delivery Platforms in a Hybrid Distance Education Program. Am J Pharm Educ. 2009,73(5):95. doi:10.5688/aj730595.
  » https://doi.org/10.5688/aj730595
• ³⁶
  Rochester CD, Pradel F. Students' perceptions and satisfaction with a web-based human nutrition course. Am J Pharm Educ. 2008,72(4):91. doi:10.5688/aj720491.
  » https://doi.org/10.5688/aj720491
• ³⁷
  Ruggeri K, Farrington C, Brayne C. A global model for effective use and evaluation of e-learning in health. Telemed E-Health. 2013,19(4):312-21. doi:10.1089/tmj.2012.0175.
  » https://doi.org/10.1089/tmj.2012.0175
• ³⁸
  Salter SM, Vale S, Sanfilippo FM, Loh R, Clifford RM. Long-term effectiveness of online anaphylaxis education for pharmacists. Am J Pharm Educ. 2014,78(7):136. doi:10.5688/ajpe787136.
  » https://doi.org/10.5688/ajpe787136
• ³⁹
  Sancho P, Corral R, Rivas T, González MJ, Chordi A, Tejedor C. A blended learning experience for teaching microbiology. Am J Pharm Educ. 2006-70(5):120. doi:10.5688/aj7005120.
  » https://doi.org/10.5688/aj7005120
• ⁴⁰
  Ward C, Garrett S, Marsh W. Does instructor presence in the classroom influence examination scores in a therapeutics course delivered via interactive video-conferencing technology? Pharm Educ. 2006,6(2):119-23. doi:10.1080/00207170600665071.
  » https://doi.org/10.1080/00207170600665071
• ⁴¹
  Zlotos L, Power A, Hill D, Chapman P. A Scenario-Based Virtual Patient Program to Support Substance Misuse Education. Am J Pharm Educ. 2016,80(3):48. doi:10.5688/ajpe80348.
  » https://doi.org/10.5688/ajpe80348

Publication Dates

• Publication in this collection
  25 Nov 2019
• Date of issue
  2019

History

• Received
  28 Mar 2018
• Accepted
  01 Feb 2019