Mapping the scientific research on the negative aspects of the medical school learning environment

1. Institute of Psychiatry, University of São Paulo, São Paulo, SP, Brasil 2. Pontifical Catholic University of São Paulo, Sorocaba, SP, Brasil 3. Family and Community Medicine, Southern Illinois University School of Medicine, Carbondale, Illinois, USA 4. Department of Medicine, Johns Hopkins School of Medicine, Baltimore, Maryland, USA 5. School of Medicine, Federal University of Juiz de Fora, Juiz de Fora, MG, Brasil


Mapping the scientific research on the negative aspects of the medical school learning environment
Rodolfo F. Damiano, MD 1 Andrey O. da Cruz, MS 2 José G. de Oliveira, MS 2 Lisabeth F. DiLalla, PhD 3 Sean Tackett, MD, MPH 4 Oscarina da Silva Ezequiel, MD, PhD 5 Giancarlo Lucchetti, MD, PhD 5

INTRODUCTION
Environment can be defined as "the surroundings or conditions in which a person, animal, or plant lives or operates". 1This concept, derived from the biological sciences, has increasingly been the subject of study in medical sciences and educational research.Specifically for medical schools, a student's "surroundings or conditions" encompass physical, social, and psychological influences and must be conducive to developing the knowledge, attitudes, skills, and behaviours students will need to practice as physi-cians.The terminology used to describe this environment varies, and has included educational environment, 2 teaching environment, 3 and, most commonly, the learning environment. 4he first study that focused on studying learning environments (LE) in higher education dates back to 1958. 5 Concerning medical schools' LEs, Hutchins (in 1961) 6 developed the first questionnaire and the first attempt to understand quantitatively how the environment might impact students' attitudes, values, REVIEW ARTICLE and behaviours.Since then, many tools have been developed to assess medical students' perceptions of their LE, 7 showing the growing importance of this subject to medical researchers.According to Cohen,8 if medical schools intend to deal with the erosion of professionalism during the course of medical training, "purging their own learning environments of unprofessional practices" (p.610) is a key endeavour.
Much research now supports Cohen's 8 idea that this purging of negative aspects of the LE is critical for developing a professional physician.Much of this focuses on the cognitive/curricular or social aspects of the LE and was derived from small populations of students. 9Reported perceptions of poor LE have already been correlated with high levels of student burnout and worse perception of quality of life, 10 decreased personal growth, 11 worse academic performance on the United States Medical Licensing Examination Step 1, 12 and also less time spent by students on activities involving direct patient contact. 13owever, many LE general instruments used in research might not encompass all aspects of the LE that influence students' lives.][16][17][18] Mapping the research on the negative aspects of the LE will help to identify areas that are already well explored, areas where more work needs to be done, how interests have trended over time among researchers, and the geographic and cultural areas where interest in these topics is greatest.Thus, the purpose of this study was to build on our current understanding of LEs through a comprehensive bibliometric analysis that develops a broader framework for LEs and their negative aspects, which can guide further investigation about each specific topic (such as original studies and systematic reviews) and medical curricula interventions.

METHODS
From September 2016 to January 2017, we carried out a bibliometric analysis to evaluate all original articles related to medical schools' LE up to December 31 st , 2016.][21] Since this is a review of literature, ethical approval is not required for this project.The sequence of the main phases is described below.

KEYWORDS SELECTION
Initially, three authors (R.F.D., A.O.C., J.G.O.) reviewed a sample of LE articles and independently generated a list of keywords focusing on capturing all potentially negative aspects of medical school LEs.After each list was created, four authors (R.F.D., A.O.C., J.G.O., G.L.) examined the list to eliminate redundancies and add new words to the list.Then, each independent researcher (R.F.D., A.O.C., J.G.O.) created a list of themes (clusters) made up of related keywords and compared them with the Learning Environment literature to check for alignment with the current scientific data.For this initial stage of development of themes, we examined systematic reviews and the most prominent articles published in high tier journals.
One author (R.F.D.), in collaboration with the institution librarian, was responsible for comparing each list of themes, merging similar ones and removing duplicates.A discussion among all authors brought a consensus of twelve thematic clusters, including the "general LE" and eleven negative aspects of medical schools' LE: hidden curriculum (negative), unethical behaviours, bullying/hazing, violence, sexual discrimination, homophobia, racism, social discrimination, minorities discrimination, professional misconduct, and other negative aspects.These clusters were arbitrarily defined by the authors of this paper.The general theme was based on articles with a focus on the Learning Environment, usually containing "Learning Environment" in the title and using LE measurement instruments.However, the negative aspects included research that did not always define the negative aspect as a component of the LE. Figure 1 summarizes the conceptual framework of the LE used by this manuscript.

ELIGIBILITY CRITERIA
Inclusion criteria were: original studies (longitudinal studies -cohort and case-control, cross-sectional studies, case reports, experience reports and experimental studies) carried out with medical students and related to their LE.Studies considered out of area (not related to LE) and out of population (not with medical students) were excluded.Furthermore, as our focus was on original studies, reviews, replications, theoretical pieces, articles without abstracts (because we needed the abstracts in order to review the paper), and book chapters were not included.No language limit was applied.

Databases search
We conducted a search in six electronic databases (PubMed/Medline, Web of Science, Cochrane Library, ERIC-ProQuest, SCOPUS, and PsycINFO) including all studies published up to December 31, 2016.A variety of Boolean expressions based on the twelve thematic clusters were created to guide the search in these databases (see Supplemental Material 1), and then each database outcome was exported to Mendeley Desktop version 1.17.6 (a free reference management program -ELSEVIER®) and sorted alphabetically in order to facilitate the review process.A hand search of references from the oldest articles identified one additional article 6 that was included in our analysis.

Data collection
Three reviewers (R.F.D., A.O.C., J.G.O.) independently screened the title, authors, and abstracts to determine if they met inclusion and exclusion criteria.If excluded, the reason for exclusion as described in the previous section was noted, and if included, the classification into one of the twelve clusters as defined above was noted.Papers that mentioned more than one theme mentioned above were discussed by all authors, who came to consensus on the most relevant finding of the article.Any discrepancies were resolved by a discussion among the reviewers in a follow-up meeting.The intra-class correlation between reviewers was assessed for the first 100 studies, based on the choice of inclusion/exclusion criteria, and we found an intra-class correlation coefficient of 0.915, showing excellent reliability between the three reviewers.

Bibliometric analysis
All included articles were exported to Excel for Mac version 14.7.2 (Microsoft®), and then each article was classified according to its characteristics: title, authors' name, journal title, journal's impact factor (by Web of Science, 2015), study design, year of publication, article's number of citations (by Web of Science and Google Scholar), and country of origin (of the corresponding author).Then descriptive statistics of all variables were analysed.

RESULTS
We found 9,337 articles across 6 databases and 1 article via hand search, resulting in a total of 9,338 articles (see Supplemental Material 2).After dropping duplicates (using the automatic Mendeley function), 5,155 articles remained.Based on our eligibility criteria, 4,445 articles were withdrawn due to one  Supplemental Material 3 shows the distribution of publications of all articles related to medical schools' LE included in this manuscript.The first such publication dates back to 1961; 6 publications remained relatively stable and infrequent until the 21 st century, when there was a notable increase in manuscripts related to medical schools' LE.In fact, 80% of all articles related to medical schools' LE were published after 1999.Notably, a large spike occurs at approximately 2006; almost 70% of all articles were published from 2006 through 2016.
The characteristics of these studies are shown in Table 1.Most of them (77.6%) are cross-sectional and quantitative studies (67.4%).We found only 41 (5.8%) experimental studies and 75 (10.5%)longitudinal studies.Almost two-thirds of the articles were published in journals with an impact factor (IF) greater than 1.00, with one third published in journals with IF greater than 3. Finally, Table 2 presents the 20 most-cited articles in the area of medical schools' LE.The number of citations for these articles is quite high, with 6 articles having more than 100 citations in Web of Science (WoS) and 10 articles having more than 200 citations in Google Scholar.
Supplemental Materials 4-7 present the characteristics of each of the 12 themes defined by this article.The journal Academic Medicine has published the most articles on medical students' LE across most of the dimensions we examined.Whereas researchers from the United States published most of the papers in this field, Pakistan published the most in bullying/ hazing.Finally, some areas are quite new in the scientific literature, and until the 1980s there were no published articles on the following areas: unprofessional behaviours, violence, hidden curriculum (negative), unethical behaviour, racism, homophobia, and bullying/hazing.
In relation to the instruments used to measure the LE, we found high use of the following tools: the Dundee Ready Education Environment Measure (DREEM), 2 the Medical Student Learning Environment Scale (MSLES), 22,23 and the most recent Johns Hopkins Learning Environment Scale (JHLES). 4omparing articles using these tools (cluster: learning environment -general) with all articles, we found an even higher proportion of manuscripts published between 2011-2016 (53%) that used these popular measurement tools.

DISCUSSION
This study represents the first comprehensive bibliometric description for learning environment (LE) research in medical schools with a focus on negative aspects.We identify here the relatively recent rapid increase in interest in LE globally, demonstrating in- creased awareness of the importance of this topic, and also the recent attempts to improve LE research study designs.For educators, this is an important call to increase our attention to the importance of the medical school LE as well as to explore in greater depth the potentially negative aspects of medical school LEs.
To facilitate our understanding about the conceptual framework of the LE area, we developed a concept map (Figure 1) based on the authors' own experience and the most prominent studies reviewed by this manuscript.First, our understanding is that both positive and negative aspects might influence students' perception of their LE.In this article, we decided to focus more on the negative ones.We have identified ten main negative areas that might impact students' perception of their LE (in parentheses the most cited on each area): unprofessional behaviors; 17 sexual discrimination; 24 minorities discrimination; 25 violence; 26 hidden curriculum (negative); 27 unethical behaviors; 28 racism; 28 homophobia; 30 bullying/ hazing; 31 and social discrimination. 32These negative themes may be concomitantly the source and the consequence of a "bad" LE; that is, these negative environments may create a poor perception that creates a positive feedback on these unacceptable behaviours.Educators should be aware of this, attempting to prevent a state where the "mean" turns inherent and cannot be seen, such as in the famous study of the Stanford prison experiment. 33oteworthy is that while there has been an increasing number of articles published in medical education in general, not all content areas receive the same attention. 34In our study, we found a recent and rapidly growing interest in medical school LE research, with the great majority (approx.70%) of studies having been published from 2006 onward and in journals with an IF greater than 1.00, further indicating interest in this field among medical education researchers.What may be driving this interest in the medical school?First, our data suggest that the availability of LE measurement instruments may facilitate research.Over half of the published articles that utilized the most frequently used measures were published since 2011.Second, interest is likely being driven by greater concern for the potential negative impact of poor learning environments in medical schools, particularly student mistreatment and overwork, as these impact trainees' empathy and well-being. 10,34-36 Third, medical school accreditation standards may play a role, as the creation of a new Liaison Committee on Medical Education 39 (LCME) standard related to LE quality coincides with the rapid increase in LE studies.
The United States has by far the most LE research identified in this study, followed by the United Kingdom, Canada, Australia, and the Netherlands.These findings are similar to analyses of all medical education articles 40, 41 and global scientific production as a whole.Given that medical education research on the learning environment is dominated by several countries that do not represent most of the world's medical schools, educators and investigators must be cautious to ensure that they focus on local needs and do not seek to overgeneralize their results.For example, in the field of empathy research, a decline in student empathy during medical schools seems to have been largely driven by several studies from the U.S., and they were not corroborated by the bulk of international studies. 42At the same time, when reviewing the top countries by topic, we saw that some developing countries had the greatest number of publications for certain clusters.For example, Pakistan had the most publications on bullying/hazing and Nigeria had the third most in violence.Future studies should explore whether these findings reflect the initiative of individuals or research teams or a larger trend in negative LE aspects.Considering that, like most medical education research, cross-sectional and single site studies comprised the majority of LE research, future work should use multiple methods across cultures to better understand the complex interactions between students and their LEs.
Our findings have implications for health managers and medical educators, providing further evidence that the LE could have a positive but also a negative influence on medical students.Educators must promote appropriate infrastructure, active learning strategies, good clinical scenarios, high-and low-fidelity labs, formative feedback, valuable educational content, and consistent theoretical and practical assessments.Yet, on the other hand, educators must be aware that unethical behaviours, bullying, violence, sexual discrimination, professional misconduct, and the hidden curriculum could impair medical students' academic performance and mental health.Thus, they must become aware of these possible be-haviours occurring in their curricula.The early identification of these negative aspects of the LE and the implementation of educational and preventive interventions should serve to minimize medical training distress and future unprofessional behaviours.
This research has several limitations.First, we focused only on original studies to characterize the current evidence base related to negative aspects of the LE, which meant that we excluded many other highly cited reviews and theoretical studies that may be influential in this field.Second, we may not have identified every relevant original research article, as no database has every paper and no search strategy can find every paper.However, we did not limit by language, so this is truly an international search.Third, the databases that we searched tended to be more focused on American and European journals, possibly excluding important contributions from the southern hemisphere.Fourth, this is a novel operationalization of the learning environment and has limitations and potential biases; however, it is important to initiate a discussion about all these possible important aspects of the LE.Fifth, many of the articles included more than one of the negative areas identified in this study.However, we tried to isolate the most important subject of each manuscript in order to facilitate our understanding.Finally, we focused more on negative aspects of the learning environment.Positive aspects, such as role models, peer relationships, and sense of engagement, clearly are important in understanding medical students' LE.However, we concentrated on negative aspects because the focus of this paper was to identify research that is related to negative outcomes for medical students and that has potential for remediation.In the future, it will be important to incorporate both negative and positive aspects of the medical school LE that contribute to changes in students' empathy and well-being.

CONCLUSION
Our analysis identifies the most important areas that articles, authors, and countries have studied or reported on in terms of negative aspects of the learning environment in medical schools.We demonstrated an important growth of scientific production in high-quality journals, especially in the 21 st century .However, more studies are needed that investigate the negative aspects of medical students' LE, with particular attention to experimental and cross-cultural/multi-school studies.Heightened awareness of negative aspects of the medical student LE should be useful in empowering medical professionals to make changes in the LE that will, in turn, improve student professionalism.
00. Academic Medicine (IF 4.194), Medical Education (IF 3.369), Medical Teacher (IF 2.355), BMC Medical Education (IF 1.312) and Teaching and Learning in Medicine (IF 1.159) represent five leading journals in this area.Most articles on this topic had corresponding authors who resided in the United States (43.5%), followed by the United Kingdom (8.6%) and Canada (5.5%).

FIGURE 1 .
FIGURE 1. CONCEPTUAL FRAMEWORK OF LE USED IN THIS ARTICLE.

TABLE 1 .
CHARACTERISTICS OF STUDIES ON MEDICAL SCHOOLS' LEARNING ENVIRONMENT