Evaluation of the effectiveness of an Internet-based continuing education program on pharmacy-based minor ailment management : a randomized controlled clinical trial

1Department of Pharmaceutical Sciences, Faculty of Pharmaceutical Sciences of Ribeirão Preto, University of São Paulo, Ribeirao Preto, São Paulo, Brazil, 2Department of Pharmaceutical Sciences, State University of Londrina, Londrina, Paraná, Brazil, 3Department of Pharmacy, Universidade Federal de São João Del Rei, Divinópolis, Minas Gerais, Brazil, 4Department of Clinical Pharmacology, University Hospital Dubrava, Zagreb, Croatia, 5Department of BSI, Faculdades Integradas de Ourinhos, São Paulo, Brazil


INTRODUCTION
Minor ailments are defined as "common or selflimiting or uncomplicated conditions which can be diagnosed and managed without medical intervention" (e.g., cold, cough, and indigestion) (Pray, 2005).Minor ailments are responsible for a substantial part of healthcare expenditures.For example, up to 18% of the general practice workload in the United Kingdom (UK) is estimated to be related to minor ailments, at a cost of £2 billion annually (Scotland, 2003).Eight percent of emergency department consultations each year involved minor ailments, costing the UK National Health Service (NHS) £136 million annually (Bednall et al., 2003;Morris, 2001).Three in five Americans report having taken at least one nonprescription drug in the past 6 months (NCPIE, 2013).
Individuals often make community pharmacies their first, and sometimes only, healthcare visit for minor ailments (Kamat, Nichter, 1998;Hoa, 2011).In the UK, pharmacy-based minor ailment schemes (PMASs) provide public access to NHS treatment and/or advice via a community pharmacist (CP), pharmacy personnel, or, where appropriate, referral to other healthcare professionals (Bellingham, 2005).These schemes were originally proposed by the UK NHS as part of their long-term strategy to encourage patient self-care and the utilization of pharmacies as the first port of call for minor ailments where professional support was required (Department of Health, 2000).A systematic review by Paudyal et al. (2013) verified that minor ailments are being dealt with appropriately by PMASs, with low reconsultation rates and high symptom-resolution rates.
However, studies performed in several other countries showed that CPs frequently do not appropriately assess the patient's condition, do not choose the correct therapeutic approach, do not provide objective information about medications or how to use them, or assess patient outcomes with the use of medications (Mesquita et al., 2013;Ratanajamit, Chongsuvivatwong, 2001;Kubiliene et al., 2006) thus raising concerns about therapeutic and adverse clinical consequences for patients.
Pharmacy education curricula in several developing countries do not focus on patient care, with only a few disciplines that focus on developing communication skills, semiology, and pharmacotherapy knowledge (de Castro, Correr, 2007;Ghilzai, Dutta, 2007;Volmer, Vendla, Vetka, 2008).This gap in pharmacy graduation curricula in developing countries can result in inappropriate CP knowledge, skills, and competency and hinder pharmacy-based minor ailment management.Continuing education (CE) programs are vitally important for expanding basic pharmacy education and enhancing therapeutic management skills, particularly in areas where insufficient training has been received or achieved during undergraduate studies (International Pharmaceutical Federation, 2006), such as pharmacy-based minor ailment management.
Internet technology is an emerging option for the provision of CE programs because it offers practicing professionals the opportunity to continue in the workforce while engaging in an active-learning environment, allowing them to proceed at their own pace and at their convenience (Means et al., 2009).However, in the area of pharmacy-based minor ailment management, few teaching and learning models have been developed for Internetbased technology, especially in developing countries.
Previous studies focused on CE programs for the management of major chronic illnesses, such as asthma, depression, and diabetes, used live media methods (e.g., workshops, lectures, and case studies), and were performed in developed countries (Rouleau, Beauchesne, Laurier, 2007;Saini et al., 2006;Chen et al., 2004;Pham et al., 2013).The differences between minor ailments and major chronic illness, between Internet-based and live media methods, and between pharmacy education curricula in developing and developed countries do not permit extrapolation of published data to our area of interest.Thus, we developed and evaluated the effectiveness of an Internet-based CE program on pharmacy-based minor ailment management.

Study design and setting
This study was approved by the Research Ethics Committee of the Faculty of Pharmaceutical Sciences of Ribeirao Preto, University of Sao Paulo, Brazil, under Protocol number 407.645.This was a randomized controlled clinical trial developed from March 10, 2014, to May 10, 2014, in community pharmacies in the Assis and Ourinhos microregions of Brazil.We developed a 2-month Internet-based CE program on pharmacy-based minor ailment management.The effectiveness of the Internet-based CE program on pharmacy-based minor ailment management was evaluated using a structured questionnaire before and after the intervention.The Assis and Ourinhos microregions cover 29 municipalities and have an estimated population of 544,000 that is served by a total of 185 community pharmacies.Brazilian community pharmacies are private healthcare establishments where drugs, medical products, and other products (e.g., toiletries and beauty products) are dispensed, with or without a prescription (depending on the specific legislation for the product).
In Brazilian community pharmacies, legislation necessitates the presence of at least one pharmacist who is responsible for the community pharmacy at all times.The establishments must also list on their employee rosters the number of pharmacists who are required to conform to this legislation.To assume the responsibility of a community pharmacy, an individual must have a degree in pharmacy from an institution that is accredited by the Brazilian Ministry of Education and must be registered with the Regional Board of Pharmacy in the state where he or she practices (Brazil, 1973).
The Brazilian National Health Surveillance Agency (ANVISA) classifies drugs as prescription drugs and nonprescription drugs (Brazil, 2003).Nonprescription drugs can be acquired by the patient without interactions with any healthcare professional (e.g., physician, CP, or nurse).Prescription and nonprescription drugs can be sold in pharmacies.However, despite the existence of legislation that regulates the sale of drugs in Brazil, it is still very common to obtain prescription drugs in Brazilian community pharmacies without presenting a prescription, and several types of markets (e.g., bars, small markets, and diners) sell certain nonprescription drugs (e.g., analgesics, antipyretics, and antacids).

Study population
Community pharmacists were eligible for inclusion in the study if, during the study period, they were employees of a participating community pharmacy that was registered with the Regional Board of Pharmacy of Sao Paulo State and worked during business hours (Monday to Friday, 8 AM to 5 PM).
Every community pharmacy in the aforementioned microregions was visited by four researchers during business hours.These researchers were pharmacy school graduates and members of our research team.The CPs did not receive any compensation for participating in the study.The researchers contacted the CPs in each establishment to invite them to participate in the study.The researchers explained to the CPs that the objective of the study was to develop and evaluate the effectiveness of an Internetbased CE program on pharmacy-based minor ailment management.Information regarding the online CE program and study protocol was also provided.The participants were assured that the data gathered would be anonymous and kept strictly confidential.The CPs could refuse to participate at any time.Community pharmacists who agreed to participate in the study were evaluated for eligibility criteria, and some of their general characteristics were recorded (i.e., age, sex, function in the pharmacy, duration of experience as a CP, and hours worked per week).

Sample size
Every eligible CP who agreed to participate in the study was included in our sample, and there was not sample size calculation.

Randomization
JMP 8.0.1 software (SAS, Cary, NC, USA) provided computer-generated random sequences that allocated CPs randomly to the intervention and control groups according to the Regional Board of Pharmacy registration numbers of the selected CPs.

Description of the interventions
Community pharmacists who were assigned to the intervention group participated in a 2-month Internetbased CE program on pharmacy-based minor ailment management and answered three different questionnaires that assessed their reaction, learning, and behavior before and after the CE program.
Community pharmacists who were assigned to the control group had no access to the Internet-based CE program and answered two different questionnaires that assessed learning and behavior only once (i.e., before the intervention group completed the CE program).

Internet-based CE program development
As proposed by some researchers (Hussainy et al., 2010;Meštrović et al., 2012;Fitzgerald et al., 2009) the curricular design of our Internet-based CE program was developed using two consecutive research methods: a survey of CPs' educational needs and a literature review.
To assist with identifying the pharmacy-based minor ailment management educational needs of the CPs, reviewing the literature, and elaborating the CE program, an Expert Reference Group was formed.The Expert Reference Group comprised researchers with expertise in pharmacy-based minor ailment management and experience in teaching disciplines of interpersonal communication, semiology, and pharmacotherapy.Figure 1 shows an overview of the stages of creating the CE program.
The survey of CPs' educational needs evaluated topics in the management of minor ailments about which CPs have doubts or are insecure in providing clinical services to patients.The survey was conducted in September 2013 and consisted of qualitative (in-depth interviews) and quantitative (close-ended) questions according to the procedures recommended by Hawe et al. (1990).One researcher of our team visited community pharmacies in the study microregions and randomly applied the survey to a convenience sample of 50 CPs.The literature review was performed by the Expert Reference Group to gather and assess information on the CPs' educational needs identified in the survey.After the literature review and evaluation of the results of the CP survey, the Expert Reference Group convened five rounds of discussions until they reached a consensus regarding the curricular design of the CE program.The Expert Reference Group attempted to design a curriculum that comprised the primary information on pharmacy-based minor ailment management and the most common and relevant doubts that CPs had in this area.
The goal of the program was to provide CPs with the knowledge necessary to manage the care of patients who presented with minor ailments in community pharmacies.Step 1: Participants accessed the CE program webpage and downloaded printable material developed by the Expert Reference Group.The participants had to read this material.The participants could read this material when he or she wanted and as many times that he or she wanted.

•
Step 2: The participants accessed the CE program webpage and downloaded a supplementary file that contained material (e.g., clinical guidelines, legislation, and manuscripts) that was selected by the Expert Reference Group.The participants had to read this material.The participants could read this material when he or she wanted and as many times that he or she wanted.

•
Step 3: The participants accessed the CE program webpage and downloaded a questionnaire regarding the topic addressed in the module.The questionnaire was composed of two open-ended questions.The participants should answer and submit the questionnaire on the CE program webpage.The participants could take as long as he or she needed to answer the questionnaire.

•
Step 4: The participants accessed the CE program webpage and participated in an online forum regarding the topic addressed.Each forum had one moderator (i.e., one researcher of our team who had expertise in pharmacy practice; each group of 10 participants was supervised by one moderator).
In the forum, the participants could discuss their doubts with the moderator and other participants.
The participant could participate in the forum as long as he or she wanted during a period of 6 days for each module.The five modules were developed to take 20 h to complete.Time limitations were not specified, thus allowing the participants to complete the CE program at their leisure while at home or at work.The participants were instructed that steps 1-3 should be completed in 6 days, and step 4 should be completed in another 6 days, for a total of 12 days for each module and 60 days for the entire CE program.

Pretesting the Internet-based CE program
Pretesting the Internet-based CE program with a group of CPs was needed to assess the acceptability of the educational program and its suitability to the needs of the target population (Hawe, Degeling, Hall, 1990) The pretesting was conducted in three stages as proposed by Aslani et al. (2006).First, we conducted individual discussions with a convenience sample of 10 CPs.Draft copies of the written protocol were distributed to the pharmacists for review and comment, and they were asked to provide feedback on the informational content of the protocol and its feasibility and practically in community pharmacy practice during individual discussions with a member of our research team.All of the discussions were tape-recorded and transcribed verbatim, and the content was analyzed.
In the next stage (group discussion), the amended protocol was mailed to a convenience sample of CPs (n = 5) who had previously completed the mail survey in the needs assessment stage.The pharmacists were asked to assess protocol implementation in their practice and provide feedback on the same issues as the subjects who participated in the individual discussions.
For the final stage (pilot test), a convenience sample of 10 CPs completed our online CE program.They were requested to comment on the feasibility and practicality of the Internet-based CE program for community pharmacy practice, provide their opinions, and suggest any possible strategies to improve the program.
After evaluating the pretest, the Expert Reference Group made the necessary changes and developed the final version of the Internet-based CE program.The results of the pretest indicated that the Internet-based CE program was found to be useful overall, easy to read and understand, and of an appropriate length and informational content.Only a few modifications were needed.None of the CPs who participated in the pretesting were involved in the next step of the study.

Evaluation of the effectiveness of the Internetbased program
The effectiveness of our Internet-based CE program was evaluated using the modified version of Kirkpatrick's Model for Summative Evaluation (Bloom, 2005).This method measures four different areas: Level 1: learner satisfaction (reaction) Level 2: learner outcomes (learning) Level 3: performance (behavior) Level 4: patient or health outcomes (results) For the purposes of the intervention, we decided to evaluate Levels 1, 2, and 3. To evaluate learner satisfaction (Level 1) with the CE program, a questionnaire was developed with a semantic differential 5-point scale (1 = poor, 5 = excellent).The questionnaire was composed of 10 questions regarding the CPs' impression of the Internet-based format, ease of use, appropriateness of content, relevance of content to practice, amount of information, duration of the CE program, provision of further reading material and references, appropriateness of exercises, appropriateness of tutorial activities, and overall impression of the CE program.The questionnaire was answered by the participants after finishing the CE program.The control group did not answer this questionnaire because they did not access the CE program and so were unable to give their impressions about it.
To evaluate learner outcomes (Level 2) of the Internet-based CE program, we tested the knowledge of the study participants.The knowledge test was developed and validated by the Expert Reference Group and underwent a pilot test.The test assessed knowledge in the following areas: nonprescription drug legislation (n = 2 questions), assessment of patients who present with minor ailments (n = 2 questions), decision-making process in the management of minor ailments (n = 3 questions), and selection of treatment for minor ailments (n = 3 questions).The knowledge test consisted of 10 close-ended questions.Correctly answering each question earned one point, with a maximum possible score of 10 points.The questionnaire was administered to the intervention group before (pretest) and after (posttest) the Internet-based program.The content of the questionnaire was the same each time, but the order of the items varied.The questionnaire was administered just once (March 2014) to the control group.The participants answered the questionnaire online, and their identification was not revealed.
To evaluate performance (Level 3) with the CPE program, we developed a self-evaluation questionnaire.Each item of the questionnaire was scored on a 5-point scale.Higher scores represented more positive behavioral practices related to minor ailments that are considered important for patient care.The questionnaire was administered to the intervention group before (pretest) and after (posttest) the Internet-based program.The content of the questionnaire was the same each time, but the order of the items varied.The questionnaire was administered just once (March 2014) to the control group.The participants answered the questionnaire online, and their identification was not revealed.

Internet-based tools used for the CE program
A website with a moodle platform was developed by our Expert Reference Group with technical support from a Brazilian university (Faculdades Integradas de Ourinhos).The moodle platform guaranteed free access to the study participants via a personal login and password.The moodle platform enabled the participants to access the CE program content, answer questionnaires, and participate in discussion groups (forums).

Intervention group study protocol
Community pharmacists who were assigned to the intervention group received a personal login and password to access the Internet-based CE program.Before beginning the Internet-based CE program, the CPs had to answer a pretest questionnaire.After completing the pretest questionnaire, the CPs accessed the Internetbased CE program and completed the curriculum over 2 months (from March 10, 2014, to May 10, 2014).After completing the curriculum, the CPs answered the posttest questionnaire and satisfaction survey (Level 1).

Data analysis
Simple descriptive statistics were used to describe the following general characteristics of the CPs: age, sex, function in the pharmacy, duration of experience as a CP, and hours worked per week.The effectiveness of the Internet-based CE program on learner satisfaction was evaluated based on the scores the CPs gave to the CE program.The effectiveness of the Internet-based CE program on learner outcomes and performance was evaluated by comparing the pretest and posttest scores in the intervention group and by comparing the posttest scores in the intervention group and the scores in the control group.Student's independent-sample t-test was used to analyze the data.Values of p < 0.05 were considered statistically significant.The results are expressed as the median and corresponding interquartile range, mean ± standard deviation (SD), or proportion.Numerical variables were tested for a normal distribution using the Kolmogorov-Smirnov and Shapiro-Wilk tests.The analyses were performed using Statistica 12.0 software (StatSoft, Sao Caetano do Sul, SP, Brazil).

General characteristics of the community pharmacists
Of the 185 CPs contacted, 121 (65.4%) agreed to participate.A total of 61 CPs were assigned to the intervention group, and 60 CPs were assigned to the control group.Thirty-three CPs in the intervention group and 51 CPs in the control group completed the study protocol (Figure 2).The general characteristics of the CPs were similar between groups (Table I).The CPs were young adults, with an average of 8 years of experience as CPs.More than 64% of these professionals did not have administrative functions and engaged in patient-focused activities during most of their work time (Table I).
Learner satisfaction (Level 1) with the CE program was high for every point evaluated.Most of the CPs rated the program as ≥ 4, with no CP rating the program as ≤ 2 (Table II).
Significant improvement in learner outcome scores was observed in the intervention group after the conclusion of the CE program.Despite the significant improvement in the learner outcome scores, most the CPs in the intervention group did not assign scores > 6 on a scale of 0-10.The intervention group gave higher scores than the control group (Table III).
Performance significantly improved after the CE program.Most of the CPs in the intervention group had scores > 4. The intervention group had higher scores than the control group (Table IV).
According to the principles of adult learning, learning should be self-directed, collegial, and reflective (Frankford, Patterson, Konrad, 2000;Edwards, Hansen, Raggatt, 2002).None of these alone is sufficient.The higher scores given by the CPs for learner satisfaction suggest that our CE program allowed these processes, suggesting that the Internet-based format was highly acceptable and appears to be a feasible mode of educating CPs.This is consistent with the findings of previous studies that reported positive responses with regard to Internet-based CE programs (Stafford et al., 2010;Walters et al., 2012;Zolezzi, Blake, 2008).Internet-based CE programs are a viable educational intervention strategy for CPs.However, the high dropout rate (i.e., almost half of our sample did not finish the CE program) suggest that our CE program may require changes in its format (e.g., duration, time to complete the modules).We may need to extend the time allotted to complete each module because CPs work ~ 44 h per week, with little time that can be dedicated to studying.Based on the experience of our research team, we suggest introducing a 1-day workshop (live intervention) that focuses on the practice setting to improve the motivation of the CPs and reduce the dropout rate.According to Frankford et al. (2000), healthcare professionals are highly motivated to learn if the learning is perceived as useful, particularly in the short term, and if it occurs within practice settings (Frankford, Patterson, Konrad, 2000).Therefore, the introduction of practice setting activities during a workshop could be an effective strategy.The introduction of a 1-day workshop may not be difficult for CPs who are enrolled in the CE program because the workshop can be convened on the weekend.
Despite the improvement in learning outcomes in CPs who completed the CE program, the CPs' scores were still insufficient.A growing body of evidence, primarily from the medical education literature, indicates that CE programs can be more successful with regard to both learning and changes in practice if more than one intervention is applied and are continuing not opportunistic (Mazmanian, Davis, 2002;Davis et al., 1995;Leigreid et al., 2010).The implementation of a CE program over a longer period of time may be an effective strategy for improving learning outcomes because the CPs will have more time to read the material and participate in the educational activities.
Performance significantly improved after the CE program (mean total score = 4.8 [0.2] out of a maximum of 5.0).Our CE program promoted more positive behavior directed toward the CPs' practices related to minor ailments that are considered important for patient care.
The number of people accessing the health care systems is increasing, while the number of physicians available to care for these patients is decreasing; so the physician-centered health care model will not sustain this demand for health services.This situation leads health care models around the world undergo profound and sensitive changes.One of the health care delivery models that had recently gained traction to help address this is team-based care.By practicing in a team-based care model, physicians and other clinicians (e.g.CPs, nurses) can care for more patients.Community pharmacies in countries such Australia and UK have evolved to offer several health-related services like health-promotion work (e.g.displaying posters and counselling/advice), screening disease services, PMASs (including the authorization CPs prescription of some drugs) and drug therapy management (Benrimoj, Roberts, 2005;Noyce, 2007).Recently the Brazilian Federal Board of Pharmacy published legislation that regulates the provision of health-related services in the pharmacy and the prescription of some kind of drugs by the CPs (Brazil, 2013a;Brazil, 2013b); the PMASs is one of the health-related services regulated by this legislation.In 2014, the publication of the law number 13,021 by the Brazilian Government finally recognized the community pharmacies as health-establishment in Brazil (Brazil, 2014).The law number 13,021 states that is responsibility of the CPs to provide health-related services in the community pharmacies (Brazil, 2014).However, besides these legislations the practice of health-related services in CPs is still scarce (Mesquita et al., 2013;Obreli-Neto et al., 2013).We can say that nowadays is a propitious moment to better qualify the CPs in the provision of health-related services like PMASs; and Internet-based is an interesting tool to provide CE to the CPs given the large land area of Brazil (8 514 876 km², Brazil is the fifth larger country in the world) and lack of time of the CPs (in our study the mean work time per week was higher than 44 hours).

Strengths and limitations
Our study design was a controlled clinical trial, which is the most powerful design to assess the effectiveness of interventions.The randomization of our sample resulted in two groups with similar general characteristics, knowledge, and practices in the management of minor ailments, thus minimizing potential bias in the sample characteristics.Another strength of our study was the sample selection.Every CP who was considered eligible and wanted to participate in the study was included in our study, which improved the generalizability of our results.The CPs who were enrolled in the present study were employees of different community pharmacies.Some of the CPs were employees of large pharmacy chains, and others were employees of small community pharmacies.This reduced possible bias associated with being employees of the same community pharmacy chain.Such bias could reduce the generalizability of the results to other CPs.The evaluation of the effectiveness of our Internet-based CE program used several levels of the modified version of Kirkpatrick's Model for Summative Evaluation, thus allowing us to assess the effectiveness of the CE program in different areas, which is different from several previous studies that assessed only one level.
Despite these advantages, our study also has some limitations.The small sample size may reduce the generalizability of the data.However, as recommended in the literature, more emphasis should be placed on the internal validity of an educational program rather than external validity because it is important to ensure that the program is effective with the target sample before it can be extrapolated to the target population (Windsor et al., 1984).This study primarily assessed the immediate impact of an Internet-based CE program.The questionnaire was administered just after the conclusion of the CE program.
To minimize this bias, the evaluation of the impact of the CE program should be performed several months after the conclusion of the CE program (e.g., 12 months) or in an ongoing manner (e.g., evaluate the CPs every month for 1 year).Other limitation is that the questionnaire was administered once in control group, and it is possible that control group CPs could have better scores when answer the same questionnaire again; this bias could had overestimated the effect of the CE program.The evaluation of Level 3 (performance, behavior) could had be better assessed using the simulated patient (SP) method; SP method is an individual who is trained to go to a pharmacy and enact predetermined scenarios, while being indistinguishable from genuine patients.The use of questionnaire possibility that the CPs knows that they are being evaluated, this can alter the testee's performance, yielding results that do not express what actually happens in the pharmacy practice setting.The SP method has been shown to help overcome this methodological problems, and has become a useful and objective tool for evaluating professional performance (Xu, de Almeida Neto, Moles, 2012).For economic and logistical reasons, we did not include Level 4 (Patient or health outcomes) of the modified version of Kirkpatrick's Model for Summative Evaluation, and so we cannot determine whether our Internet-based CE program was effective at this level.

CONCLUSION
Our results indicated that the Internet-based CE program was a viable educational strategy for pharmacy management of minor ailments and showed positive results at three different levels of the modified version of Kirkpatrick's Model for Summative Evaluation (learner satisfaction, learner outcomes, and performance).Given the high number of patients who seek care for minor ailments in the community pharmacy setting and the insufficient knowledge and ability of CPs in the management of these health conditions, the implementation of an Internet-based CE program by national healthcare systems or pharmacist associations may improve patient care.
Assessment of patients in the management of minor ailments • Module 4: Decision-making process in the management of minor ailments • Module 5: Selection of treatment in the management of minor ailments Each of these modules was composed of four steps: •

FIGURE 1 -
FIGURE 1 -Overview of stages in creating an internet-based continuing education program on pharmacy-based minor ailments management.
The development and evaluation of new educational tools is important for improving patient care.Previous studies reported that CPs do not adequately manage minor ailments.The present results indicated that CPs had positive views about the Internet-based CE program, but we observed a high CE program dropout rate.The Internet-based CE program significantly improved learner outcomes and performance, with a significant difference between the intervention group and control group.However, despite the improvement in learner outcomes in the intervention group, the individual CP scores remained insufficient (mean total score = 6.2 [2.4] out of a maximum FIGURE 2 -Flowchart of the study.

TABLE I -
General characteristics of the community pharmacists who participated in the study

TABLE II -
Scores for leaner satisfaction (Level 1) with the continuing pharmaceutical education program (intervention group; n = 33)

TABLE III -
Learner outcome scores (Level 2) with the continuing pharmaceutical education program -sample t-test was used.Values of p < 0.05 were considered statistically significant.a Posttest scores in intervention group compared with scores in control group.b The questionnaire was administered only once in the control group.

TABLE IV -
Performance improvement (Level 3) with the continuing pharmacist education program