Evaluation of interprofessional education for shared decision making in drug therapy: a scoping review on methods and instruments

This study aimed to describe the research methods and instruments used in the evaluation of interprofessional education strategies for shared decision making in drug therapy. The types of evaluation employed were categorized according to Kirkpatrick’s adapted model. A scoping review was conducted, following the PRISMA-ScR recommendations. Among the 21 selected studies, there was a predominance of quantitative methods in the evaluation of educational experiences (n=18). Of these, the most common aspect evaluated was “students’ attitudes and perceptions towards interprofessional education and practice” by means of validated instruments (n=13). Ten different instruments were identified and found to be in line with the Interprofessional Education Collaborative competencies. The variety of instruments signals the growing production of knowledge about this topic, but points to the challenge of conducting comparative analyses between educational experiences around the world.


Introduction
Pharmacotherapy is one of the most widely used resources in current health care systems in order to prevent, maintain, and recover health status. However, the prescription and inappropriate use of drugs is the origin of great damage to people's health, requiring interventions to minimize the resulting morbidity and mortality 1 . The interaction of two or more health professionals in the decision-making process in drug therapy is necessary to improve the health outcomes of patients 2,3 .
Shared decision making happens when health professionals and patients collaborate, the best evidence is shared, and patient preferences are respected, and patients are considered effective members of the health team 4 . In this context, it is important to address shared decision making in drug therapy when training future professionals to work in interprofessional health care.
According to the World Health Organization (WHO), interprofessional education (IPE) "occurs when students from two or more professions learn about, from and with each other to enable effective collaboration and improve health outcomes" 5 (p. 7). Therefore, the ultimate goal of IPE is to ensure safe and effective health care 6,7 .
IPE initiatives have been developed in undergraduate health courses in different countries, showing positive results [8][9][10][11] . To assist in the development of these educational activities, 38 key competencies for interprofessional collaborative practice were defined and organized by the Interprofessional Education Collaborative (IPEC), subdivided into four domains: values and ethics; roles and responsibilities; interprofessional communication; and team and teamwork 12,13 .
Given the importance of promoting IPE and in order to know more about how interprofessional educational experiences have been implemented, it is also necessary to know how the evaluation of these activities has been conducted in research. In this article, the term evaluation is adopted as the process of gathering evidence that allows judging the effectiveness and value of an educational activity 14 .
Peltonen et al. 15 conducted a scoping review on existing instruments for measuring interprofessional collaboration in healthcare, with a focus on in-service professionals, without a specific look at the evaluation of students-in-training participating in IPE activities.
Based on the above, a scoping review was conducted by our research team to identify studies that describe and evaluate IPE experiences involving aspects of decision making in drug therapy 16 . The present study is a result of this review and has as specific objectives to describe the research methods and instruments employed in the evaluation of such educational experiences from the students' perspective, as well as to analyze quantitative instruments assessing "students' attitudes and perceptions toward interprofessional education and practice" regarding collaborative key competencies 13 .

Methodology
A scoping review was conducted, as this type of study is suitable for mapping the available literature in a given research area. All relevant literature is included, regardless of methodological quality, since the objective is precisely to present an overview on a given topic 17,18 .
The review protocol was developed using the methodological framework proposed by the Joanna Briggs Institute 18 and was registered in the Open Science Framework (osf.io/kfy27). This review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) 19 .

Systematic search and formulation of the research question
A search strategy was prepared using the acronym PCC, where: "P" refers to population; "C" to concept; and "C" to context 18 , with the aim of identifying primary studies that describe and evaluate IPE experiences involving pharmacotherapy decision-making. The results obtained in this review originated two studies; the first, already published, focuses on the teaching and learning approaches used in this context 16 . In this second study, the articles were analyzed to answer the following question: what are the research methods and instruments used to evaluate these experiences in IPE for decision making in drug therapy, from the student's perspective?
The search process was conducted in seven different databases: MEDLINE (PubMed), Excerpta Medica Database (EMBASE), Cochrane Library, PsycInfo, Education Resources Information Center (ERIC), Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Latin American and Caribbean Health Sciences Literature (LILACS). For each database, a specific strategy was built with the MeSH descriptors or their corresponding DeCs descriptors in English and Portuguese, and the combination of them with their synonyms (search strategies available on the Open Science Framework platform: osf.io/kfy27).
The manual search was conducted by checking the references of all included studies and searching for the last 10 years in the three journals that obtained the highest number of articles retrieved in the search: Journal of Interprofessional Care, Currents in Pharmacy Teaching and Learning, and American Journal of Pharmaceutical Education.

Eligibility Criteria
The following inclusion criteria were used: primary studies with all methodological designs that answered the research question; articles written in English, Portuguese or Spanish, with no restriction regarding publication date. Exclusion criteria: review articles; articles whose population was not composed of at least two or more health and social care undergraduate students; articles that did not address the experience of IPE regarding decision making in drug therapy.

Study selection, data extraction and analysis
The articles identified in the search, after removing duplicates, were gathered in the Rayyan® software 20 . The article selection process was conducted by four independent researchers working in pairs (Kirla Barbosa Detoni and Ariane Lopes André; Cristiane de Paula Rezende and Bárbara Taciana Furtado), in two stages: (1) title and abstract screening; (2) full-text reading. Discrepancies were resolved by consensus and discussion with a fifth reviewer (Simone de Araújo Medina Mendonça).
The data extracted from the selected articles were organized in an Excel® spreadsheet developed by the reviewer team. The reviewer pairs extracted the data independently. In a later step the team jointly discussed the collected information and updated the spreadsheet. Data were collected on the following aspects: a) characteristics of the study (country of origin and year of publication); b) research objectives; c) research methods and instruments used to evaluate the educational experiences; and d) validation of quantitative instruments in the original language and in Brazil.
The results were synthesized in narrative form and a descriptive analysis was conducted, determining absolute and relative frequency measures. The types of evaluation employed in the studies were categorized according to the model adapted from Kirkpatrick 21 .
The adapted Kirkpatrick model is a slightly modified version of the original model 22 , expanded with the addition of two evaluation items on levels 2 and 4, which has been suggested by experts in the field of IPE 14 . This adapted model presents the following levels of evaluation: reaction, which includes the participants' perspective on the educational experience (level 1); modification of students' attitudes and perceptions (level 2a); acquisition of knowledge and skills (level 2b); behavior change, which includes the application of learning in practice (level 3); organizational changes (level 4a); and benefit to patients/users (level 4b).
The contents of the validated quantitative instruments used to evaluate the modification of students' attitudes and perceptions regarding education and interprofessional practice were analyzed according to the collaborative competencies proposed by IPEC 13 . For this purpose, the full version of these instruments was searched. The items of each instrument were analyzed by the main author and categorized into one of the four domains of IPEC key competencies: values and ethics; roles and responsibilities; interprofessional communication; and team and teamwork. The categorization was done by comparing the content of the instrument items with the description of the competencies and sub-competencies in the official IPEC document (2016 version) 13 . In the following step, meetings were held with Kirla Barbosa Detoni and Simone de Araújo Medina Mendonça for collaborative analysis of the categorization of these items.

Results and Discussion
The search retrieved 5000 publications. From them, 1261 were excluded due to duplicity. A total of 103 articles were selected for full reading, with 42 resulting from the database searches and 61 from the manual search. 21 studies were included in this scoping review ( Figure 1). The general characteristics of the studies are described in Frame 1. To explore the feasibility of joint therapeutic teaching with medical and pharmacy students and to evaluate the students' educational experience.
Quantitative research.
Application of a post-educational intervention questionnaire.
Quantitative instrument without specific name, developed by the authors -Contains seven objective items, evaluated in relation to the agreement or disagreement of the students.
Stewart et al. 25  To describe the first year of an interprofessional experience in geriatrics and to analyze the reflective feedback of pharmacy students about the responsibilities of other professions in relation to geriatric patients in general.
Qualitative research. The students were asked to answer reflective questions about the topic.
There was no questionnaire application. The answers were evaluated by thematic analysis.
To design and implement pediatric prescribing workshops and assess changes in students' knowledge, skills and attitudes by comparing 9 interprofessional workshops with 10 non-interprofessional workshops.
Quantitative research. Application of the validated questionnaire pre and post intervention. To assess the change in attitudes of pharmacy and medical students towards physician-pharmacist collaboration after participating in an IPE event at a community service.
Quantitative research. Application of a validated questionnaire pre and post intervention. To assess students' attitudes about interprofessional collaboration and confidence regarding medication management competencies after participating in an interprofessional module.

Scale of Attitudes Toward
Application of a validated questionnaire pre and post intervention. At the end of the instrument, questions related to the acquisition of skills in medication management were added. The year of publication of the studies ranged from 1995 to 2020, with 90% (n = 19) of them published after 2010, which shows that interest in the subject has increased in recent years. This review also showed the lack of studies on IPE published in the southern hemisphere. Almost all of the research was conducted in North America (n = 10, 47%) and Europe (n = 9, 43%), as well as Asia (n = 2, 9%) and Oceania (n = 1, 4%).
Concurrent with the results found in this study, other reviews of the literature indicate that most research in IPE present evaluations referring to levels 1-2b, which focus on outcomes related to student learning in the short term 9 . Overall, these results are appropriate for educators and educational and professional organizations. However, these authors point out that more research is needed to evaluate the changes that IPE can bring in actual clinical practice settings (level 4), including organizational changes and benefits to patients and users, as these outcomes may be more appropriate for national stakeholders such as managers, policy makers, and regulators. The U.S. Institute of Medicine also reinforces the need for research to assess the connection between IPE and its impact on practice, including the impact on patient and population health and outcomes for the health care system as a whole 2 .
Two studies used exclusively qualitative evaluation instruments 26,41 and seven employed both qualitative and quantitative evaluation methods 25,28,30,31,36,39,43 . The qualitative data collection methods applied in these studies included: open-ended questions included at the end of quantitative instruments (n = 5) 25,28,31,41,43 ; group reflective feedback (n = 2) 26,37 and focus groups (n = 1) 36 . Hardisty et al. 30 used multiple data collection methods, including observation, interviews, and focus groups. The qualitative evaluation methods employed were punctual and did not reach the complexity and depth typical of qualitative research. These results, therefore, highlight the need for further and more robust qualitative research to better understand the processes and relationships involved in providing interprofessional educational activities.
All studies that conducted quantitative research to evaluate educational activity employed cross-sectional study design with the application of a structured questionnaire (n = 18). Of these, 16 studies conducted evaluation before and after the educational intervention, and two conducted the questionnaire application only after the intervention 23,24 . In fact, among the tools for evaluating IPE activities, structured questionnaires have been the most used 44 , possibly because of their easiness of application and data analysis.
Reeves et al. 14 published in 2015 a guide to improve the quality of evaluations of IPE initiatives in order to generate more robust evidence. The authors point out that regardless of the type of research (quantitative or qualitative), essential questions should be taken into consideration when planning the evaluation, such as defining the purpose, based on the learning context; the conceptual framework of the project; the academic level of the students, and the formulation of the question that is intended to be answered with this specific evaluation. The authors also highlight the importance of identifying and engaging different stakeholders in the evaluation process.
This review showed that validated quantitative instruments are widely used to evaluate educational experiences. In sum, 12 instruments presented as validated were identified in these studies. Two of them aim to assess students' "acquisition of specific clinical knowledge and skills" regarding opioid overdose 34 and acquired immunodeficiency syndrome (AIDS) 43 . Ten instruments aimed to evaluate the modification of students' attitudes and perceptions regarding interprofessional education and practice, namely: The ten instruments highlighted above were previously validated in the English language literature, with the exception of the one used by Chua et al. 39 , which was developed and validated by the authors themselves. The instruments: AITCS II, IEPS, ISVS, RIPLS and SATP2C were validated for Brazilian Portuguese. The variety of the identified validated instruments shows the growing production of knowledge on the subject. On the other hand, it calls the attention to the challenge of performing comparative analyses between interprofessional educational experiences around the world. It was not the objective of this study to analyze the quality of the validation processes of the instruments.
The items encompassed by the validated instruments that assessed the modification of students' attitudes and perceptions regarding interprofessional education and practice, highlighted above, were assessed by the authors according to the four domains of collaborative key competencies proposed by IPEC, as described in the methodology (Frame 2).  *The items present in the instruments that were not identified by the authors as belonging to one of the four domains of IPEC collaborative competences were grouped as "Others" and refer to general issues, such as satisfaction with the educational experience, recommendation of the activity for others students and preference between different teaching methodologies.
The domain "team and teamwork" was present in all the instruments evaluated, corresponding to approximately half of the items in the AITCS II (46%), T-TAQ (50%) and RIPLS (53%) scales, and representing 60% of the items in the SPICE scale. The domain "roles and responsibilities" was also contemplated in all instruments and predominated in SATP2C (67%), being less important in T-TAQ (10%) and SAIL-10 (10%).
The domain "interprofessional communication" corresponded to half of the items covered in the ICCAS instrument and approximately 30% in the ISVS and T-TAQ. Items related to this domain were not identified in the SPICE and SATP2C instruments. The domain "values and ethics" was not identified in three instruments (SPICE, ISVS and ICCAS), and the IEPS scale showed the highest proportion of items related to this domain among the instruments evaluated (28%).
The items present in the instruments that were not identified by the authors as belonging to one of the four domains of IPEC collaborative competencies were grouped as "others". The aspects evaluated by these items refer to general issues such as satisfaction with the educational experiences, recommendation of the activity to other students, and preferences among different teaching methodologies.
The analysis performed on the instruments showed that their items fit the competences described by IPEC, and most of the items evaluated were concentrated in the domains: "team and teamwork" and "roles and responsibilities". The domain "values and ethics", in general, was the least evaluated in the instruments. The less prominent group of collaborative competences in the instrument items reflect a logic of care that still focuses mainly on technical issues, which is a strong feature of the training of health professionals. It is important to think about interprofessional work from a perspective that can be used to equalize the power relations between patients and professionals, and also among health professionals. Interprofessionalism should seek to horizontalize relationships, promote better and greater communication with ethics and values such as respect and dialogic capacities.
The tools presented great variability with regard to the domain of the key competence evaluated. Among the tools analyzed, RIPLS has the most items related to the "team and teamwork" domain (53%) and STAP2C has the most items involving the "roles and responsibilities" domain (69%). ICCAS is the instrument that most contemplated the "interprofessional communication" domain, and IEPS the one that most contemplated the "values and ethics" domain. In this sense, we hope that this analysis will help researchers and teachers in the selection of instruments that are more assertive according to the competence that is the focus of that specific educational experience.
This scoping review had some limitations. Some of the included studies provided sparse information, which made data extraction difficult. This issue highlights the need to improve the quality of descriptions in future articles, but at the same time reflects the reality of the publications. Although no gray literature search was performed, this review relied on an extensive literature search in seven electronic databases, in addition to the manual search. As a limitation of the scope review methodology itself, no analysis of the quality of the included studies was performed, since the objective was to present a map of the literature on the subject. Therefore, it was not possible to establish whether the methods and assessment tools employed by the studies were adequate for the interprofessional teaching and learning processes described.

Conclusion
This study allowed the identification of the methods and instruments used in the evaluation of experiences in IPE for decision making in drug therapy, revealing the predominance of the use of quantitative methods. A large number of validated quantitative instruments were identified with a focus on assessing the modification of students' attitudes and perceptions regarding interprofessional education and practice. Analysis of these instruments, according to the domains of key competencies described by IPEC, showed that the domains of "Team and Teamwork" and "Roles and Responsibilities' were predominant. The variety of instruments indicates the growing production of knowledge on the subject, but points to the challenge of conducting comparative analysis between educational experiences around the world.