Evidence of validity for a Health Teaching Skills Scale

Evidências de Validade para uma Escala de Habilidades de Ensino em Saúde

Evidencia de validez para una escala de Habilidades de Enseñanza en Salud

Abstract

There are few studies found in the literature that analyze the university professor duties, specifically in the area of health. The purpose of this article was to describe the construction of a scale to evaluate the importance of health teaching skills and to seek evidence of validity using factorial (exploratory and confirmatory) analyzes based on the internal structure. The sample was composed by university students of Nursing and Medical courses (n = 315). The results indicated the presence of 4 factors that remained in the model after performing the confirmatory factorial analysis, which are: use of information and communication technologies in teaching (13 items, α = 0,87); Teaching in practical health scenarios (10 items, α = 0,81); Interpersonal relationship (6 items, α = 0,89); and Didactics (5 items; α = 0,63). We concluded that the instrument has initial evidence of validity and may be applicable in studies to diagnose the learning needs of university professors.

Keywords:
university; students; nursing; medical school; factor analysis

Resumo

Há poucos estudos encontrados na literatura que analisem o contexto do professor universitário, especificamente na área de saúde. Esse artigo teve como propósito descrever a construção de uma escala de avaliação da importância das habilidades de ensino em saúde e reunir evidências de validade utilizando análises fatoriais (exploratória e confirmatória) com base na estrutura interna. A amostra foi composta por estudantes universitários dos cursos de enfermagem e medicina (n = 315). Os resultados indicaram a presença de quatro fatores que permaneceram no modelo após a realização da análise fatorial confirmatória. São eles: uso de tecnologias de informação e comunicação no ensino (treze itens; α = 0,87); ensino em cenários práticos de saúde (dez itens; α = 0,81); relacionamento interpessoal (seis itens; α = 0,89); didática (cinco itens; α = 0,63). Concluiu-se que o instrumento possui evidências iniciais de validade e pode ser aplicável em pesquisas de diagnóstico de necessidades de aprendizagem de professores universitários.

Palavras-chave:
universidade; estudantes; enfermagem; medicina; análise fatorial

Resumen

En la literatura son escasos los estudios encontrados para analizar el contexto de los profesores universitarios, específicamente en el área de salud. Este artículo tuvo como propósito describir la construcción de una escala para evaluar la importancia de las habilidades docentes en salud y buscar evidencias de validez utilizando análisis factoriales (exploratorio y confirmatorio) basados ​​en la estructura interna. La muestra se compuso por estudiantes universitarios de los grados de Enfermería y Medicina (n = 315). Los resultados indicaron la presencia de cuatro factores que permanecieron en el modelo después de la realización del análisis factorial confirmatorio. Son ellos: uso de las tecnologías de la información y de la comunicación en la enseñanza (trece ítems, α = 0,87); enseñanza en entornos prácticos de salud (diez ítems, α = 0,81); relación interpersonal (seis ítems, α = 0,89); didáctica (cinco ítems, α = 0,63). Se concluye que el instrumento tiene evidencias iniciales de validez y puede ser aplicado en investigaciones para diagnosticar las necesidades de aprendizaje de los profesores universitarios.

Palabras clave:
universidad; estudiantes, enfermería; medicina; análisis factorial

Introduction

Several studies report that the university professor, due to the new demands of education, needs to face the challenge of following the transformations of society, seeking strategies of permanent and lifelong education that enable him or her to follow the speed of a teaching context, supported by sophisticated technologies. These technologies expand people’s access to relevant knowledge and give a continuous character to learning, making it possible anytime and anywhere with mobile devices (Álvarez-álvarez, 2015; Merchán & Díaz, 2015; Pereira, Loiola & Gondim, 2016; Sánchez, Rodríguez, Barrios & Miranda, 2016).

According to Soares and Cunha (2010), the teaching career is mainly restricted to academic education focused on research, such as master’s and Ph.D. courses, to the detriment of training actions aimed at teaching and university extension. In the context of health, this scenario is even more critical, since incomplete teacher training can result in a poorly qualified performance for teaching complex skills (e.g., conducting care procedures, analyzing laboratory tests, communication, working in interdisciplinary teams), in the classroom, in practical contexts, and in critical care and assistance to patients (Abbad et al., 2016; Mikkonen et al., 2018).

The scarce pedagogical qualification of health care professors contributes to the maintenance of an old and traditional model of professional training, unable to meet the needs of society in the context of the Brazilian Unified Health System (SUS), or the standards stipulated by the World Health Organization (2013), which value quality and patient safety. Abbad et al. (2016) point out several educational strategies adopted in contexts of Health Education (HE), and the Horizon Report - Higher Education Edition (2017) describes the main trends in higher education in several countries around the world, despite the lack of consensus on the skills and qualifications for teaching in the area (Mikkonen et al., 2018). Allied to this, changes in society and pressures on the provision of quality health care services have further boosted the emergence of new teaching methods, which can be used by professors in different contexts (classroom and real or simulated contexts), enabling the combined learning of several domains, such as cognitive, affective, and psychomotor (Causby, Reed, McDonnell, & Hillier, 2018; Steinert et al., 2006).

Current education models have adopted active learning methodologies, such as digital media, augmented reality, gamification, realistic simulations, excerpts from television series, and other technological support resources. These methodologies favor the use of practical, interactive, and dynamic activities that enable experimentation and collective reflection on innovative problem solutions, compatible with the work of health professionals and teams. One of the barriers to the use of active methodologies by university professors is the lack of knowledge and teaching skills that enable the creation of innovative and attractive HE strategies for students (Abbad et al., 2016; Coyne et al., 2018; Horizon Report - Higher Education Edition, 2017; Jerrentrup et al., 2018).

Abreu and Moura (2003) point out the importance of building instruments capable of reliably measuring the real learning needs of the teaching public in order to achieve positive impacts on teacher training programs. The study by Mendoza Jacomino et al. (2010), which analyzed the Cuban context of teacher training in Nursing, noted that the essence of developing the skills of nurses is directly linked to learning opportunities in practical and real situations. According to Hou et al. (2011), who developed several instruments to assess the effectiveness of clinical teaching in China, there is still a shortage of clinical instructors properly trained to teach practical training in health care, in the field of Nursing, when compared to teachers with general teaching skills and inherent to theoretical classes. Steinert et al. (2006) also state that, for years, there were those who believed in the assumption that a good doctor would naturally be a good professor. This concept has changed radically, and today it is recognized that the preparation for teaching and the development of skills to exercise this profession is continuous and of crucial importance in medical education.

A study by Liberali, Novack, Duke, and Grosseman (2018) showed that 64.20% of accredited Brazilian medical schools receive formal training in communication skills for medical students. However, there are still many challenges for implementing the teaching of these skills in Brazil, mainly due to the lack of tools with evidence of validity to conduct a consistent assessment of this type of teaching (Catani et al., 2018). The lack of instruments that include specific health items in different contexts (classroom and practice environment) constitutes a relevant scientific and academic gap.

Given this scenario, the question is: What factors should be considered when assessing teaching skills? How should these factors be measured? Who will evaluate them? Therefore, caution is needed when constructing and defining teaching assessment tools. According to Jara-Gutiérrez and Zapata-Castañeda (2015), in the area of ​​health care, it is not recommended to apply the same model of pedagogical evaluation to teachers who work in very different areas, such as clinical and basic sciences. Despite the high risk of social desirability, the teacher’s self-assessment model in the diagnosis of the teaching skills necessary for its development should be valued, given the possibility for the teacher to reflect critically on aspects that favor or hinder their work. On the other hand, several studies reiterate that this process of evaluating teaching skills must be elaborated from several different sources of evaluation, such as students, course coordinators, and peers (De Almeida, 2017).

Scientific publications on the training of university professors are rather scarce and restricted, being limited to the discussion on the need for reflection on the development of a more active attitude of the teacher, and critical discourse on reinventing the role of this professional in higher education institutions. However, it is observed that this situation has been changing in recent years with the growing interest of researchers on the subject (Burgess & McGregor, 2018; Pereira et al., 2016).

Assessment tools for teaching skills

A review of the scope of national and international articles, published between 1990 and December 2018, was conducted in the research platforms for the Coordination for the Improvement of Higher Education Personnel (CAPES) and the Scientific Electronic Library Online (Scielo). A total of 53 articles addressed teaching evaluation models and instruments on teaching skills; however, 45 presented the measurement scales in the article and only 12 of these were in the health area. Table 1 presents the main content dimensions identified in the literature about these 12 specific instruments in the health area.

Table 1
Content dimensions of instruments on teaching skills in the health field

Content dimensions	Authors/years
Interpersonal relationships, communication, and respect for diversity	Nehring (1990); Kotzabassaki et al. (1997); Makoul (2001); Lee et al. (2002); Maunye et al. (2009); Coniel Linares et al. (2014); Si, J. (2015); Gaspard e Yang (2016); Catani et al. (2018).
Teaching/didactic skills	Nehring (1990); Kotzabassaki et al. (1997); Rozendo et al. (1999); Lee et al. (2002); Salminen et al. (2013); Coniel Linares et al. (2014); Si, J. (2015).
Domain of health-specific content	Nehring (1990); Kotzabassaki et al. (1997); Lee et al. (2002); Salminen et al. (2013); Coniel Linares et al. (2014); Si, J. (2015).
Assessment and Feedback	Nehring (1990); Kotzabassaki et al. (1997); Rozendo et al. (1999); Lee et al. (2002); Salminen et al. (2013); Si, J. (2015).
Teacher's personality	Kotzabassaki et al. (1997); Lee et al. (2002); Salminen et al. (2013); Si, J. (2015).
Planning	Rozendo et al. (1999).
Active methodologies (role-playing, group discussion sessions, case studies)	Maunye et al. (2009); Hou et al. (2011).
Clinical skills in health practice settings	Salminen et al. (2013); Gaspard e Yang (2016); Hou et al. (2011).

• Source: Produced by the author.

Table 1 shows a prevalence of more general content dimensions, which are applied in different teaching contexts, such as the relationship with the student, to the detriment of those specific to higher education. In the last 10 years, there has been a greater incidence of instruments that specify dimensions focused on HE, such as clinical skills in practice settings, in addition to the presentation of items that describe the use of active technologies and methodologies as HE strategies.

In general, it was found that the 12 studies mentioned reported concern with the development of professors and students in the health area, especially in Medical and Nursing courses. However, these instruments present unclear definitions of content dimensions, which can make it difficult to develop comparable instruments, and 41.60% did not perform procedures to assess the psychometric properties of the instruments.

The twelve instruments mentioned have similarities in terms of the content dimensions identified. Thus, it is possible to make comparisons of the instruments with other studies in the area of HE. The model proposed by Galvis (2007), for instance, suggests 4 types of skills needed by university teachers in the 21st century: (1) Intellectual skills - focused on the domain of content; (2) Intra and Interpersonal - related to rapport with students and self-motivation; (3) Social - related to the ability to communicate clearly and logically, in addition to the ability to negotiate with instances of society, such as the State; and (4) Professional - focused on innovative teaching strategies, use of technologies and assessment techniques. Most of the skills mentioned in Galvis’ model (2007) are represented in content dimensions of other instruments identified in the literature, especially interpersonal skills, considered crucial for teachers working in the health area.

The systematic review conducted by Mikkonen et al. (2018) did not identify studies that assessed the 3 competency indicators: knowledge, skills, and behaviors in health care professors. Most studies have described teaching skills in the form of items that assess the teacher’s didactics and personality.

Given the social and academic relevance of the topic presented, this article aims to describe the construction and search for evidence of validity based on the internal structure of a Scale for the Assessment of the Importance of Health Teaching Skills (EAIHES). It also specifies the process of exploratory factor analysis (EFC) and confirmatory factor analysis (CFA) of the scale and, finally, discusses its final format, its contributions, and limitations to the research area.

Method

This is a study of a mixed nature, with qualitative and quantitative steps and descriptive, exploratory, and correlational character. The next sections describe the 2 studies that composed the development and the evidence of validity of the instrument.

Study 1- Instrument development and content validation

Initially, a literature search was performed to measure instruments that described teaching skills. A total of 45 instruments and 423 items were found in the literature on the evaluation of university teaching, but only 12 focused on HE. After excluding items that described similar and redundant teaching skills, a list was made up of 226 items classified in 13 dimensions, which included generic and health-specific teaching skills. There was also a translation, adaptation, and back-translation of items from English and Spanish to Portuguese, totaling a list of 55 items.

Content validation of the first version of EAIHES, consisting of 55 items, was performed in order to assess which content dimensions belonged to each item, as well as the structure, clarity, and suitability of expressions (Borsa & Seize, 2018). The content dimensions were composed of Planning, Teaching Strategies, Teaching in real and simulated settings, Mastery of communication and information technologies, Assessment of Learning, and Relationship with the student.

The judges were 6 university professors, one with a background in Education, 3 in Nursing, and 2 Medical professors. All teachers were experts in the subject of HE and with experience in developing measuring instruments. Subsequently, one judge was discarded, who showed a very divergent understanding from the other specialists, in order to minimize possible opinion conflicts. Only items that obtained 80% of the judges’ agreement on their adequacy to the dimension were kept in the instrument, and suggestions from the judges regarding the organization and writing of the items were added, totaling 34 items for the next stage.

For semantic validation of the instrument, in order to evaluate the instructions and items regarding the understanding of the language used, 12 students from the Nursing and Medical courses (6 from Bahia and 6 from the Federal District) and 6 professors from the same courses (2 from Bahia and 4 from the Federal District) participated. There was 100% agreement of the participants as to the adequacy and understanding of the instrument’s language.

Study 2 - Evidence of validity based on internal structure

Participants

The sample, chosen by availability, was composed of 315 students from the Nursing and Medical undergraduate courses from the cities of Salvador and Brasília. Among the respective sample, 91% of students fall in the age group between 18 and 25 years old; 7%, between 26 and 33 years old; 1.3%, between 34 and 41 years old; 0.3%, between 42 and 49 years old; and 0.3 %, between 50 and 58 years old. Also, 59.7% of these students are taking a nursing degree and 65.4% are studying at private higher education institutions. Thus, the minimum of 100 cases was exceeded, and the ratio of 10 cases per item required for this type of analysis (Hox, Moerbeekm & Van de Schoot, 2017; Mertler & Reinhart, 2016).

Instrument

The version of the EAIHES, resulting from Study 1, consisted of 2 parts, the first consisting of 34 objective questions (statements about teaching skills), in addition to an open-ended question with optional completion. The second part contained 6 questions related to the students’ sociodemographic and academic data. The items were answered on an 11 - point Likert scale (ranging from 0 to 10), where 0 (zero) corresponds to “not important” and 10 (ten) to “very important”.

Proceedings

After authorization from the higher education institutions (2 private and 2 public), the EAIHES was administered in person, at the beginning or end of one of the classes, when participants were asked to complete an Informed Consent Form (ICF), in 2 copies, one for the participant and the other collected by the researcher. The ICF described the process of submission and approval of the research by the Research Ethics Committee of the School of Health of UnB, via Plataforma Brasil, in compliance with Resolution No. 466/2012, of the National Health Council (approved under number 2.312.046).

Data Analysis

The data were submitted to descriptive analyzes (mean, standard deviation, minimum and maximum values, frequency) and tests of recommended assumptions for exploratory factor analysis (Field, 2013; Hox, Moerbeek & Van de Schoot, 2017). The analysis for the identification of extreme cases was performed based on the calculation of the Mahalanobis distance, as well as the analysis based on the Z score and by graphic methods (Box Plot and Q-Q Plot). The MPlus statistical modeling program, version 7, was used to analyze the factorial structure of the instrument. The method of factor extraction and main axis factoring (Principal Axis Factoring - PAF) was used by the maximum likelihood method (maximum likelihood - ML), as well as the factorability of the data matrix, using the Barrlett’s Sphericity Test and Kaiser-Meyer-Olkin (KMO) value. To identify the number of factors, the following criteria were adopted: eigenvalues ​​greater than 1, minimum of 3% of variance explained by factor (Harman, 1967), inspection of the sedimentation graph (Scree Plot), besides the interpretability of the factor.

Horn’s Parallel Analysis (HPA) was also performed to compare the empirical values provided by the PAF and the random values obtained by means of Factor software, version 10.8.04. The PAF method with oblique rotation (direct oblimin) and internal consistency analyses were used, based on the Cronbach’s alpha reliability index. Finally, to check the factorial structure of the scale, Confirmatory Factor Analysis (AFC) was also performed, with WLSMV (robust weighted least squares) estimator.

Multiple fit quality indicators were also estimated: the Comparative Fit Index (CFI) (Bentler, 1990) and the Tucker-Lewis Index (TLI) (Tucker & Lewis, 1973), for both values above 0.90 are desirable; in addition to the non-significant Chi-square index (Tanaka 1987; West, Taylor & Wu, 2012). Standardized Root-Mean Residual (SRMR) (Jöreskog & Sörbom, 1981) and Root Mean Square Error of Approximation (RMSEA) (Brown, 2006) were also calculated, for which values below 0.08 are expected (Brown, 2006).

Results

The preliminary analysis by EAIHES indicated the presence of 315 cases, of which 299 were valid. It was found that the percentage of omitted responses did not show systematicity, ranging from 1% to 6%. No univariate and multivariate extreme cases identified were identified either.

The Kolmogorov-Smirnov test indicated that the data differ from a normal distribution (p = 0.001), with 100% of the items with teaching skills having a negative asymmetry format. It was verified that the linearity and multicollinearity assumptions were not violated.

It was observed that the correlations had low values, being more than 50% lower than 0.50, but not lower than 0.30. However, the KMO value was 0.87. The Bartlett’s Sphericity Test showed a value of 5826.83 and p = 0.001, suggesting, therefore, the rejection of the null hypothesis, i.e., there are differences between the correlational matrix and the identity matrix. The values of the commonalities were also analyzed and extreme values close to zero or 1 were not identified, reinforcing that there are no problems regarding factorability.

The first factorial solution suggested an empirical structure composed of 5 factors, but it was not possible to interpret it theoretically. Factors formed by items that assessed teaching skills and interpersonal relationships were already expected since they are frequently mentioned in teaching instruments in the scientific literature of the area. Other possibilities of empirical structures composed of 6, 4, and 3 factors were tested. The analysis of the sedimentation graph (Scree Plot) identified that the first 4 eigenvalues stood out.

Figure 1
Sedimentation Graph (Scree Plot)

APH was conducted to confirm the suggestion of the number of factors, indicating the maintenance of 4 factors, since the fifth factor presented random eigenvalues higher than the empirical ones. Therefore, the most satisfactory empirical structure and with the greatest theoretical sense was composed of 4 factors, which together explained 48.71% of the total variance of the participants’ responses. Table 2 shows, therefore, the final set of the EAIHES formed by 34 items divided into 4 factors.

Table 2
Factorial Loads in EAIHES Exploratory Factorial Analysis

Item	F1	F2	F3	F4
24. Play videos and films that illustrate the subject’s content and discussions.	0.67
28. Use cell phone applications, tablets, or other mobile devices as resources to support teaching and learning.	0.66
20. Use several didactic and support materials related to the contents taught (e.g. texts, magazines, websites, videos).	0.65
25. Share relevant information and content with students using social networks (e.g. Blogs, Facebook).	0.65
26. Teach statistical analysis techniques using specific software.	0.65
27. Recommend academic Internet sites that enrich the content taught in class.	0.65
3. Identify situations in which the use of objects in three dimensions can facilitate the teaching and learning of health contents (virtual reality).	0.56
16. Use more than one didactic strategy in class (ex: oral exposure, clinical case studies, role playing, games).	0.54
13. At the beginning of each class, review the student’s previous knowledge about the contents to be taught.	0.51
21. Use virtual learning platforms (e.g. moodle) to promote discussions with students.	0,48
22.Use text editor (e.g. Word, Libre Office) as a resource in preparing classes and teaching materials.	0.44
6. Use high-fidelity robots/simulators for the reproduction of complex clinical procedures.	0.41
23. Use programs to create and display graphic presentations during classes (ex: Power Point, Prezi).	0.37
9. Demonstrate to students the accomplishment of patient assistance procedures (ex: physical examination, monitoring of the clinical condition and evolution of the patient, peripheral and / or deep venous puncture).		0.83
8. Conduct rounds with the students to monitor the clinical condition and the patient’s evolution.		0.80
12. Conduct discussions with students about probable patient diagnoses in a practice setting.		0.79
11. Analyze the results of laboratory tests and / or imaging with students to monitor the clinical condition and evolution of the patient.		0.67
7. Promote clinical case discussion sessions with students and the multi-professional health team.		0.66
33. Stimulate collaborative work in multi-professional teams in teaching activities.		0.63
1.Promote activities that encourage students to interact with patients, families, caregivers and other health professionals.		0.62
2. Monitor teaching activities based on situations / problems in the health scenario.		0.62
10. Supervise students while performing patient care procedures.		0.52
4. Promote technical visits or field trips to deepen the contents of the discipline.		0.52
30. Respect students’ opinions.			0.92
29. Encourage the student to freely express his or her opinion on the content of the discipline.			0.70
32. Be willing to help the student with his or her individual learning needs.			0.70
34. Willingness to accept changes in teaching activities suggested by students.			0.65
19. Provide constructive feedback to the student about his or her academic performance.			0.53
5. Demonstrate clinical procedures with mannequins or objects that represent parts of the human body.			-0.49
15. Exemplify the contents taught, discussing real clinical cases with students.				0.62
14. Explain the contents of the course in a logical sequence.				0.49
17. Perform different forms of student assessment (e.g. tests, classroom activities, teamwork).				0.42
31. Treat patients, professionals, and students indistinctly well, respecting human diversity.				0.40
18. Adjust the level of complexity of the assessments (assignments, tests, exercises) to the content taught in the subject.				0.35

• Source: Developed by the author.

The first factor, named “use of information and communication technologies in teaching”, consists of 13 items (M = 8.24; SD = 1.18), with factorial loads varying between 0.38 and 0.67 and adequate index of internal consistency (α = 0.87, λ₂ = 0.90). It aims to analyze the evaluations of Nursing and Medical students on the importance of teaching skills aimed at the proper use by the teacher of mediation technologies in the relationship with the student, as well as resources, materials, equipment, and information and communication technologies as strategies to support higher education (ex: videos, mobile applications, and social networks).

The second factor, entitled “teaching in practical health settings”, measures students’ assessments of the importance of teaching skills in real practice settings in the health field, using active methodologies, discussions of clinical cases with students, clinical rounds, analysis of test results, conducting practical activities in the presence of patients, family members, and other professionals. It consists of 10 items (M = 9.58; SD = 0.53), factorial loads ranging from 0.52 to 0.83, and internal consistency indexes within acceptable parameters (α = 0.81, λ₂ = 0, 81).

The third factor, named “interpersonal relationship”, assesses the teacher’s social and attitudinal skills, expressed in his or her interactions with students when treating them well, showing respect for their opinions, offering constructive feedback, and being willing to accept their demands and suggestions. It consists of 6 items (M = 9.54; SD = 0.58), factorial loads ranging between 0.49 and 0.92, and a satisfactory internal consistency index (α = 0.89, λ₂ = 0.90).

And the fourth factor entitled “didactic”, is composed of 5 items (M = 9.45; SD = 0.69), has factorial loads ranging from 0.35 and 0.63, and an internal consistency index with α = 0.63 and λ₂ = 0.65. The respective factor measures perceptions about the importance of teaching skills aimed at different strategies, methodologies, resources, and educational tools appropriate to the context of undergraduate health education.

Then, CFA was performed, and it was verified that the standardized regression coefficients confirmed the 4 factors proposed and that the relationship between them had lower factor loads. The loads of the items of factor 1, “use of new technologies of information and communication (NTICs) in teaching”, ranged between 0.37 and 0.67. The loads of factor 2, “teaching in health practice settings”, ranged between 0.52 and 0.83; the items of the third factor, “interpersonal relationship”, between 0.46 and 0.92; and the last factor, “didactics”, varied between 0.35 and 0.62. The adjustment indices obtained for the 4-factor model in the AFC were χ² = 1117,841, gl = 521, SRMR = 0.07, RMSEA = 0.064 (CI = 0.058 - 0.069, p = 0.001), CFI = 0.909 and TLI = 0.902.

Discussion

The factorial solution of the scale corroborated in part with the other instruments identified in the literature, especially that of Kotzabassaki et al. (1997), Lee and Williams (2002), and that of Maunye et al. (2009). The respective instruments mentioned dimensions of content composed of items with specific teaching skills in the health area, as well as didactics, interpersonal relationships, and active learning methodologies, such as role-playing. It was observed that 3 of the 4 factors presented in the empirical structure of the scale (teaching in health practice settings, interpersonal relationships, and didactics) are also mentioned in the instrument by Hou, Zhu, and Zheng (2011).

The result of the empirical structure presented is also similar to the studies by Johnsen et al. (2002) and Salminen et al. (2010), who proposed that the skills of nursing educators should be included in 5 categories: (1) nursing competence, (2) pedagogical skills, (3) assessment skills, (4) personality factors, and (5) relationship with the student. Except for the dimension “personality factors”, it is possible to observe similarities between the dimensions mentioned and the teaching skills present in the scale of this article.

The use of videos and other digital media applicable to the educational context, which characterized the first factor, is described in the studies by Cao, Ajjan, and Hong (2013) and Coyne et al. (2018) and in the Horizon Report - Higher Education Edition (2017). Coyne et al. (2018) point out, for example, that the teacher should opt for excerpts of learning videos, illustrating authentic situations of clinical reality, which serve as a support for the student’s understanding. The literature has therefore suggested that teachers need to be encouraged by universities to use NTICs in HE, as students report high satisfaction with classes that follow this model of blended learning, mediated by teaching technologies. The teacher needs to have not only disposition but also technical skills to guarantee the interactive quality of the videos in his or her classes (Cao et al., 2013; Coyne et al., 2018).

The second factor ensured skills applicable to the health context, thus converging with those found in the instrument presented by Maunye et al. (2009), which mentions sessions to discuss clinical cases and the use of active teaching methodologies, such as role-playing, as skills necessary for the teaching-learning process. Also, students best assimilate skills related to practical settings when they are accompanied by training, practice, and feedback (Burgess & McGregor, 2018; Causby et al., 2018).

The third factor “interpersonal relationship” was considered by many studies referred to in this review as one of the most important in teaching practice. All the instruments analyzed referred to this content dimension as necessary to the teacher. In the health scenario, this competence is even more valued.

Benor (2000) describes that the medical professor, for example, must be able to teach not only specific knowledge and clinical skills, but also how to manage care cases and relate to human beings in situations of pain and suffering. Another aspect that strengthens this second factor is the need for health professionals to treat patients, families, and professionals well and indistinctly, respecting human diversity. In this step, Taylor, Condry, and Cahill (2018) state that the teaching and awareness of issues such as sexual orientation or gender identity have been neglected in medical education.

The study by Jerrentrup et al. (2018), which shows that students are strongly influenced by TV series in their learning processes, points out a high critical perception of these students on the interpersonal skills of doctors, especially with their patients and peers. Catani et al. (2018) and Kaplan-Liss et al. (2018) also emphasize the relevance of this factor, emphasizing that the Brazilian guidelines indicate that teaching patient-focused communication skills is one of the main elements for achieving satisfaction and quality in health care.

The last “didactic” factor is mentioned in almost all instruments aimed at the development of university professors. The dimension also includes skills aimed at student assessment strategies, which are also valued in the instruments of Kotzabassaki et al. (1997), Nehring (1990), Rozendo et al. (1999), and Si (2015). The respective factor has items that are also linked to the dimension of interpersonal relationships, which are also mentioned in the instruments presented by Lee and Willians (2002) and Salminem et al. (2013).

The first 3 factors showed good internal consistency rates (α = 0.87; α = 0.81 and α = 0.89), since they are higher than 0.80 (Damásio, 2012). However, the internal consistency of the “didactic” factor (α = 0.63 and λ₂ = 0.65) is classified as questionable, according to Borsa and Seize (2018) and Damásio (2012). Although the minimum accepted value for this measure is 0.70, in exploratory studies values up to 0.60 can be considered, according to Hox, Moerbeek, and Van de Schoot, (2017). In addition, this result can be influenced by the sample size (Bonnet, 2002) and the small number of items in the factor (Cortina, 1993). As indicated by Cortina (1993), on scales with few items, a lower α value does not necessarily express a low correlation between items.

The time invested in the development of the items of the scale was considered crucial in this study, confirming research such as that of Muñiz and Fonseca (2008), who affirm that a deficient construction of items damages the whole psychometric structure of the scale. A concern considered important in this stage of content construction and validation was the wording of the items, that, according to Borsa and Seize (2018), need to meet the criteria of clarity, simplicity, desirability, objectivity, the item’s need to express a behavior (not being abstract), relevance, precision, variety, modality, a balance between the segments that were addressed in the items, range, typicality, credibility, and quantity of the items. Furthermore, the elaboration of constitutive definitions in the validation process by judges contributed to making the instrument applicable and with possibilities of comparison with other similar instruments.

The adjustment indices obtained for the AFC 4-factor model were considered adequate for the model (Bentler, 1990; Brown, 2006; Jöreskog & Sörbom 1981; Tanaka, 1987; Tucker & Lewis, 1973; West, Taylor, & Wu, 2012). Taking into account that the model needs to meet the statistical quality criteria, but cannot exclude the theoretical basis underlying the model, these indices reinforce the instrument’s proposal based on this factor structure (Damásio, 2012; Hu & Bentler, 1999; Mertler & Reinhart, 2016).

It was observed that the fit quality indexes show that the proposed model brings psychometric information that corroborates the factorial structure of EAIHES with 4 factors. Regarding the regression estimators and weights, it seems that the factor, “use of new technologies of information and communication in teaching” (0.60 / 0.41; p ≤ 0.000), better estimates higher education skills.

Final Considerations

This article aimed to describe the process of designing a scale of HE skills and present its evidence of validity. The EAIHES differs from other measurement instruments identified in the literature as it includes items of technical knowledge, skills, and attitudes crucial to health teaching, most of which are considered as very important by students in Nursing and Medical schools.

The objective of the study was accomplished and the factorial solution of the scale can be used in future studies. The results showed that the scale presents evidence of construct validity and internal consistency, is applicable in university teaching contexts (Nursing and Medicine courses), and allows evaluations on the importance of a relevant set of teaching skills, according to the students. One of the main academic contributions of this article, therefore, was the construction of the scale, with evidence of validity, covering items of specific teaching skills for Nursing and Medical courses. The instrument developed can serve as an input for the process of institutional self-assessment that all higher education institutions are required to develop, within the scope of the National Educational Assessment System (SINAES).

However, the instrument of the present study needs to be improved, so that the percentages of total variance explained by the factors can increase with the inclusion of more specific health items in the questionnaires. As previously mentioned, although there are some instruments in the literature with evidence of validity about the performance of the university professor, there are few instruments that include health-specific items. Thus, we opted for the execution of the EFA in this study, given the lack of studies that present empirical evidence of their scales on the teaching skills of the university professor in the health area. Confirmatory factor validity proved the latent structure of the EAIHES four-factor model. It is worth noting, however, the scarcity of similar empirical structures identified in the literature.

It is also important to point out some limitations found in the study, to mention the perceptive character of the students when completing the scale. The nature of the measure, based only on the students’ perceptions of the importance they give to certain teaching skills, and the lack of objective indicators extracted from other sources of information (teachers, course coordinators, preceptors) or archival data sources, which could make the evidence of scale validity more robust. In future research opportunities, it is suggested the use of more objective measures to assess teaching skills, as well as the application of the instrument to other courses in the health field. Another limitation was the scoring scale chosen (0 to 10), which was not used in all its points, with a strong concentration of scores in some intervals related to the extreme positive points of evaluation. It is suggested that the method used in this research be used in future scientific investigations for the development of scales composed of more specific HE skills, such as psychomotor skills, much required in surgical procedures, and of greater complexity.

Despite their notorious importance, HE skills are still little studied in the scientific context, thus corroborating some studies mentioned in the literature review of this study. Therefore, it is relevant to continue investigating the HE skills needed by the teacher, since they directly impact the preparation of future health professionals.

We, therefore, suggest a research agenda for the advancement of knowledge about HE skills: (1) Application of EAIHES in other educational institutions and other samples of students, in order to seek new evidence of validity, considering various sociodemographic data and institutional conditions to support teaching; (2) Expansion of research on education skills focused on other courses in the health field; (3) Expansion of research on other skills of health professors in research and extension activities; (4) Test of factors with a second-order structure in factor analysis, and (5) Use of Item Response Theory for further investigations of the psychometric structure of the scale developed. The issues involving the development of university professors in the area of health have not been exhausted in this article, but they have contributed to strengthen the research dialogues in Education, Psychology, and Higher Education.

References

• Abbad, G., Parreira, C., Pinho, D., Queiroz, E., Torres, A., Furlanetto, Jorge, A., & Silva, M.N. (2016). Formação e processos educativos em saúde. Em Abbad, G., Parreira, C., Pinho, D., & Queiroz, E. (Eds), Ensino na Saúde no Brasil (pp.27-48). Curitiba: Juruá.
• Abreu, D. G., & Moura, M. O. (2014). Construção de instrumentos teórico-metodológicos para captar a formação de professores. Educação e Pesquisa, 40 (2), 401-414. doi: 10.1590/S1517-97022013005000024
  » https://doi.org/10.1590/S1517-97022013005000024
• Álvarez-Álvarez, C. (2015). Teoría frente a práctica educativa: Algunos problemas y propuestas de solución. Perfiles educativos, 37(148), 172-190. doi:10.1016/j.pe.2015.11.014
  » https://doi.org/10.1016/j.pe.2015.11.014
• Benor, D. E. (2000). Faculty development, teacher training and teacher accreditation in medical education: Twenty years from now. Medical Teacher, 22(5), 503-512. doi: 10.1080/01421590050110795
  » https://doi.org/10.1080/01421590050110795
• Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological bulletin, 107(2), 238. doi: 10.1037%2F0033-2909.107.2.238
  » https://doi.org/10.1037%2F0033-2909.107.2.238
• Bonett, D. G. (2002). Sample size requirements for testing and estimating coefficient alpha. Journal of Educational and Behavioral Statistics, 27(4), 335-340. doi: 10.3102/10769986027004335
  » https://doi.org/10.3102/10769986027004335
• Borsa, J. C. & Seize, M. M. (2018). Construção e adaptação de instrumentos psicológicos: Dois caminhos possíveis. Em Damásio, B. F., & Borsa, J. C. (Eds.), Manual de desenvolvimento de instrumentos psicológicos (pp. 15-38). São Paulo: Vetor.
• Brown, T.A. (2006). Confirmatory factor analysis. New York: Guilford Press.
• Burgess, A., & McGregor, D. (2018). Peer teacher training for health professional students: A systematic review of formal programs. BMC medical education, 18(1), 263. doi: 10.1186/s12909-018-1356-2
  » https://doi.org/10.1186/s12909-018-1356-2
• Cao, Y., Ajjan, H., & Hong, P. (2013). Using social media applications for educational outcomes in college teaching: A structural equation analysis. British Journal of Educational Technology, 44(4), 581-593. doi: 10.1111/bjet.12066
  » https://doi.org/10.1111/bjet.12066
• Catani, R. R., Valadares, E. D. S., Lacombe, J. B., Mendonça, T. M. D. S., Silva, C. H. M. D., & Paro, H. B. M. D. S. (2018). Cross-cultural adaptation of the Four Habits Coding Scheme (4HCS) for teaching and assessing patient-centered communication skills in Brazil. Cadernos de saude publica, 34, e00013918. doi: 10.1590/0102-311X00013918
  » https://doi.org/10.1590/0102-311X00013918
• Causby, R. S., Reed, L., McDonnell, M. N., & Hillier, S. L. (2018). Teaching of manual clinical skills in podiatric medicine: Theory and recommendations. Journal of the American Podiatric Medical Association, 108(2), 158-167. doi: 10.7547/15-223
  » https://doi.org/10.7547/15-223
• Coniel Linares, E., Díaz Álvarez, L. M., Hernández García, G., Lorenzo, C., Cristina, M., & Vilaú Díaz, L. Á. (2014). Necesidades de aprendizaje de los docentes de la asignatura microbiología-parasitología desde la didáctica. Revista de Ciencias Médicas de Pinar del Río, 18(3), 512-524. Recuperado de http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S1561-31942014000300015
  » http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S1561-31942014000300015
• Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98. doi: 10.1037/0021-9010.78.1.98
  » https://doi.org/10.1037/0021-9010.78.1.98
• Coyne, E., Rands, H., Frommolt, V., Kain, V., Plugge, M., & Mitchell, M. (2018). Investigation of blended learning video resources to teach health students clinical skills: An integrative review. Nurse education today, 63, 101-107.doi: 10.1016/j.nedt.2018.01.021
  » https://doi.org/10.1016/j.nedt.2018.01.021
• Damásio, B.F. (2012). Uso da análise fatorial exploratória em psicologia. Avaliação Psicológica, 11(2), 213-228. Recuperado de http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712012000200007&lng=pt&tlng=pt
  » http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712012000200007&lng=pt&tlng=pt
• De Almeida, C. S. (2017). Competências e desempenho docente: Validando escalas de autoavaliação e heteroavaliação, explorando fatores pessoais e ocupacionais associados. (Dissertação de Mestrado, Universidade Federal da Bahia, Salvador, BA). Recuperado de https://repositorio.ufba.br
  » https://repositorio.ufba.br
• Field, A. (2013). Discovering statistics using IBM SPSS statistics. London: Sage Publications Ltd.
• Galliford, N., & Furnham, A. (2017). Individual difference factors and beliefs in medical and political conspiracy theories. Scandinavian Journal of Psychology, 58(5), 422-428. doi: 10.1111/sjop.12382
  » https://doi.org/10.1111/sjop.12382
• Galvis, R. V. (2007). De un perfil docente tradicional a un perfil docente basado em competencias. Acción Pedagógica, 16(1), 48-57. Recuperado de http://190.57.147.202:90/xmlui/bitstream/handle/123456789/810/Dialnet-DeUnPerfilDocenteTradicionalAUnPerfilDocenteBasado-2968589.pdf?sequence=1
  » http://190.57.147.202:90/xmlui/bitstream/handle/123456789/810/Dialnet-DeUnPerfilDocenteTradicionalAUnPerfilDocenteBasado-2968589.pdf?sequence=1
• Gaspard, J., & Yang, C. M. (2016). Training needs assessment of health care professionals in a developing country: The example of Saint Lucia. BMC Medical Education, 16(1), 1-6. doi: 10.1186/s12909-016-0638-9
  » https://doi.org/10.1186/s12909-016-0638-9
• Harman, H.H. (1967). Modern fator analysis. Chicago: University of Chicago Press
• Horizon Report. (2017). NMC Horizon Report Preview 2017. Higher Education Edition. Retrieved from http://cdn.nmc.org/media/2017-nmc-horizon-report-he-preview.pdf
  » http://cdn.nmc.org/media/2017-nmc-horizon-report-he-preview.pdf
• Hou, X., Zhu, D., & Zheng, M. (2011). Clinical Nursing Faculty Competence Inventory - development and psychometric testing. Journal of Advanced Nursing, 67(5), 1109-1117. doi: 10.1111/j.1365-2648.2010.05520.x
  » https://doi.org/10.1111/j.1365-2648.2010.05520.x
• Hox, J. J., Moerbeek, M., & Van de Schoot, R. (2017). Multilevel analysis: Techniques and applications. Routledge.
• Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. doi: 10.1080/10705519909540118.
  » https://doi.org/10.1080/10705519909540118.
• Jara-Gutiérrez, N. P., Díaz-López, M. M., & Zapata-Castañeda, P. N. (2015). Desafíos educativos para el profesor de medicina: evaluación de su desempeño. Iatreia, 28(3), 292-299. doi: 10.17533/udea.iatreia.v28n3a07
  » https://doi.org/10.17533/udea.iatreia.v28n3a07
• Jerrentrup A., Mueller T., Glowalla U., Herder M, Henrichs N., Neubauer A. et al. (2018). Teaching medicine with the help of “Dr. House”. PLoS One, 13(3): e0193972. doi: 10.1371/journal.pone.0193972 https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0193972
  » https://doi.org/10.1371/journal.pone.0193972» https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0193972
• Johnsen, K. Ø., Aasgaard, H. S., Wahl, A. K., & Salminen, L. (2002). Nurse educator competence: a study of Norwegian nurse educators’ opinions of the importance and application of different nurse educator competence domains. Journal of Nursing Education, 41(7), 295-301. doi: 10.3928/0148-4834-20020701-05
  » https://doi.org/10.3928/0148-4834-20020701-05
• Jöreskog, K., & Sörbom, D. (1981). LISREL V: Analysis of linear structural relationships by the method of maximum likelihood. National Educational Resources.
• Kaplan-Liss, E., Lantz-Gefroh, V., Bass, E., Killebrew, D., Ponzio, N. M., Savi, C., & O’Connell, C. (2018). Teaching medical students to communicate with empathy and clarity using improvisation. Academic Medicine, 93(3), 440-443. doi: 10.1097/ACM.0000000000002031
  » https://doi.org/10.1097/ACM.0000000000002031
• Kotzabassaki, S., Panou, M., Dimou, F., Karabagli, a, Koutsopoulou, B., &Ikonomou, U. (1997). Nursing students’ and faculty’s perceptions of the characteristics of “best” and “worst” clinical teachers: A replication study. Journal of Advanced Nursing , 26(4), 817-824. doi: 10.1046/j.1365-2648.1997.00351.x
  » https://doi.org/10.1046/j.1365-2648.1997.00351.x
• Lee, W.-S. C., Cholowski, K., & Williams, A. K. (2002). Nursing students’ and clinical educators’ perceptions of characteristics of effective clinical educators in an Australian university school of nursing. Journal of Advanced Nursing , 39(5), 412-420. doi: 10.1046/j.1365-2648.2002.02306.x
  » https://doi.org/10.1046/j.1365-2648.2002.02306.x
• Liberali, R., Novack, D., Duke, P., & Grosseman, S. (2018). Communication skills teaching in Brazilian medical schools: What lessons can be learned? Patient education and counseling, 101(8), 1496-1499.doi: 10.1016/j.pec.2017.12.021
  » https://doi.org/10.1016/j.pec.2017.12.021
• Makoul G. The SEGUE Framework for teaching and assessing communication skills. (2001). Patient education and counseling , 45(1), 23-34. doi: 10.1016/S0738-3991(01)00136-7
  » https://doi.org/10.1016/S0738-3991(01)00136-7
• Maunye, T. J., Meyer, S. M., & Van Velden, C. E. (2009). An assessment of teaching strategies used by lecturers at a nursing college in Mpumalanga. Curationis, 32(3), 30-37. Retrieved from http://www.scielo.org.za/pdf/cura/v32n3/05.pdf
  » http://www.scielo.org.za/pdf/cura/v32n3/05.pdf
• Mendoza Jacomino, A., Artiles Olivera, I., & Torres Valladares, J. (2010). Demandas formativas para la concepción de la superación del profesor-tutor de enfermería, em la universidad de las ciencias médicas cubana. Revista Electrónica Educare, 14(2). Recuperado de https://www.redalyc.org/html/1941/194115606006/
  » https://www.redalyc.org/html/1941/194115606006/
• Merchán, F., & Díaz, A. (2015). La formación pedagógica del docente universitario: un reto del mundo contemporáneo. Revista Cubana de Educación Superior, 34(3), 140-154. Recuperado de http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0257-43142015000300011
  » http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0257-43142015000300011
• Mertler, C. A., & Reinhart, R. V. (2016). Advanced and multivariate statistical methods: Practical application and interpretation. Routledge.
• Mikkonen, K., Ojala, T., Sjögren, T., Piirainen, A., Koskinen, C., Koskinen, M., ... & Koskimäki, M. (2018). Competence areas of health science teachers: A systematic review of quantitative studies. Nurse education today , 70, 77-86. doi: 10.1016/j.nedt.2018.08.017
  » https://doi.org/10.1016/j.nedt.2018.08.017
• Muñiz, J., & Fonseca, E. (2008). Construcción de Instrumentos de medida para la evaluación universitaria. Revista de Investigación en Educación, 5, 13-25. Recuperado de http://reined.webs.uvigo.es/ojs/index.php/reined/article/view/40
  » http://reined.webs.uvigo.es/ojs/index.php/reined/article/view/40
• Nehring, V. (1990). Nursing Clinical Teacher Effectiveness Inventory: A replication study of the characteristics of best and worst clinical teachers as perceived by nursing faculty and students. Journal of Advanced Nursing , 15(8), 934-940. doi: 10.1111/j.1365-2648.1990.tb01949.x
  » https://doi.org/10.1111/j.1365-2648.1990.tb01949.x
• Pereira, L. M. R., Loiola, E., & Gondim, S. M. G. (2016). Aprendizagem de competências, suporte à transferência de aprendizagem e desempenho docente: Evidências de validação de escala e teste de relações. Organizações & Sociedade, 23(78), 438-459. doi: 10.1590/1984-92307856
  » https://doi.org/10.1590/1984-92307856
• Rozendo, C. A., Casagrande, L. D. R., Schneider, J. F., & Pardini, L. C. (1999). Uma análise das práticas docentes de professores universitários da área de saúde. Revista Latino-Americana de Enfermagem, 7(2), 15-23. doi: 10.1590/S0104-11691999000200003
  » https://doi.org/10.1590/S0104-11691999000200003
• Salminen, L., Minna, S., Sanna, K., Jouko, K., & Helena, L. K. (2013). The competence and the cooperation of nurse educators. Nurse Education Today, 33(11), 1376-1381. doi: 10.1016/j.nedt.2012.09.008
  » https://doi.org/10.1016/j.nedt.2012.09.008
• Sánchez, M. D. C. R., Rodríguez, M. G., Barrios, J. E. R., & Miranda, M. A. A. (2016). Validación de un modelo de competencias docentes en una universidad privada mexicana. Revista Digital de Investigación en Docencia Universitaria, 10(1), 1-15. doi: 10.19083/ridu.10.455
  » https://doi.org/10.19083/ridu.10.455
• Si, J. (2015). Needs assessment for developing teaching competencies of medical educators. Korean Journal of Medical Education, 27(3), 177-186. doi: 10.3946/kjme.2015.27.3.177
  » https://doi.org/10.3946/kjme.2015.27.3.177
• Soares, S. R., & Cunha, M. I. da. (2010). Programas de pós-graduação em Educação: lugar de formação da docência universitária? Revista Brasileira de Pós-Graduação, 7(14), 577-604. doi: 10.21713/2358-2332.2010.v7.18
  » https://doi.org/10.21713/2358-2332.2010.v7.18
• Steinert, Y., Mann, K., Centeno, A., Dolmans, D., Spencer, J., Gelula, M., & Prideaux, D. (2006). A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Medical Teacher , 28(6), 497-526. doi: 10.1080/01421590600902976
  » https://doi.org/10.1080/01421590600902976
• Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. New York: Harper & Row Collins College Publishers.
• Tanaka, J. S. (1987). “How big is big enough?”: Sample size and goodness of fit in structural equation models with latent variables. Child Development, 58, 134-146. doi: 10.2307/1130296
  » https://doi.org/10.2307/1130296
• Taylor, A. K., Condry, H., & Cahill, D. (2018). Implementation of teaching on LGBT health care. The clinical teacher, 15(2), 141-144. doi:10.1111/tct.12647
  » https://doi.org/10.1111/tct.12647
• Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1-10. doi:10.1007/BF02291170
  » https://doi.org/10.1007/BF02291170
• West, S. G., Taylor, A. B., & Wu, W. (2012). Model fit and model selection in structural Equation Modeling. In R. H. Hoyle (Ed.), Handbook of Structural Equation Modeling (pp. 209-231). Guilford Press.
• World Health Organization. (2013). Interprofessional collaborative practice in primary health care: Nursing and midwifery perspectives. World Health Organization. Retrieved from https://apps.who.int/iris/handle/10665/120098
  » https://apps.who.int/iris/handle/10665/120098

Publication Dates

• Publication in this collection
  06 Dec 2021
• Date of issue
  Jul-Sep 2021

History

• Received
  22 Jan 2019
• Reviewed
  01 July 2020
• Accepted
  01 Sept 2020