Development and Validation of an Instrument to Evaluate General Competences in Health Area Courses

ABSTRACT

The evaluation of the General Competencies in undergraduate courses in the healthcare area remains challenging.

Objectives

To develop an instrument for supporting teaching staff in evaluating the General Competencies of undergraduate students in the healthcare area; to test the reliability of the instrument with teachers and students of the same working field; validate the instrument of General Competencies, directed to teachers and students from the health area.

Methods

the present was a Methodological study, approved by REC N. 826.770. The validation of the construct, criterion and content based on the National Curricular Guidelines (NCG), and the search for a theoretical framework were performed, as well as statistical tests such as alpha Cronbach, t Test, p-value, Factorial Analysis, Pearson’s Correlation Coefficient and Akaike Information Criterion, which also ensured the reliability. The study was performed in a Higher Education Institution in Curitiba/PR, in the courses of Nursing, Biomedical Sciences, Pharmacy, Psychology and Medicine. The study population consisted of 50 evaluations of students and 50 of teachers, covering all the periods and courses during the second half of 2014.

Results

it was possible to create one model with three versions of instruments that evaluated the general competencies for the healthcare area courses. One of these was directed towards the general competencies of 10 courses in the healthcare area, subdivided in Health Care, Decision-making, Communication, Leadership, Administration and Management and Continuing Education, with one instrument directed at the student and another ‘mirror’ at the professor. The second and third versions had three dimensions: Health Care, Management in Healthcare and Education in Health, directed towards the general competencies of the new structure of the NCG of the medical course. The three versions also had an instrument for teachers and a mirror one for students.

Conclusion

To validate the instrument on General Competencies, directed to teachers and undergraduate students of health area courses, the theoretical search, the NCG and the experts’ evaluation were used for Content Validity; the t Test, Chi-square Test and Pearson’s correlation coefficient were used for Criterion Validation; statistical tests of exploratory and confirmatory factorial analysis and the AIC were used for Construct Validity; and Cronbach’s alpha and the AIC were used for Content validity, using the same steps described for the first goal, to ensure the reliability of the instruments. After this process three versions of the instrument were developed, the first two to be used with teachers and with teachers and students together; and the third version is adequate to be used with students or teachers; and also, for teachers and students together. However, it can be used by all health courses studied.

Professional Competency; Teaching; Educational Measurement; Education, Higher; Validation Studies

RESUMO

A avaliação das competências gerais nas graduações da área da saúde permanece desafiante.

Objetivos

Desenvolver um instrumento para apoiar os docentes na avaliação das competências gerais dos graduandos de cursos da área da saúde; testar a confiabilidade do instrumento com professores e estudantes do mesmo campo de atuação; validar o instrumento de competências gerais, direcionado aos professores e graduandos da área da saúde.

Métodos

A pesquisa foi do tipo metodológico, aprovada pelo Comitê de Ética em Pesquisa nº 826.770. Foram realizadas a validação de constructo, a de critério e a de conteúdo com base nas Diretrizes Curriculares Nacionais (DCNs) e na busca de referencial teórico, além de testes estatísticos como alfa de Cronbach, test t, valor de p, análise fatorial, correlação do coeficiente de Pearson e Akaike information criterion (AIC), que também garantiram a confiabilidade. O estudo foi desenvolvido em uma instituição de ensino superior de Curitiba (PR), nos cursos de Enfermagem, Biomedicina, Farmácia, Psicologia e Medicina, sendo a sua população composta de 50 avaliações de estudantes e 50 de professores, o que contemplou todos os períodos e cursos em andamento no segundo semestre de 2014.

Resultados

Foi possível a confecção de um modelo com três versões de instrumentos que avaliam as competências gerais para os cursos da área da saúde. A primeira englobou as competências gerais de 10 cursos da área da saúde, que se subdividem em: atenção à saúde, tomada de decisões, comunicação, liderança, administração e gerenciamento e educação permanente. A segunda e terceira versões contam com três dimensões, atenção à saúde, gestão em saúde e educação em saúde, voltadas para as competências gerais da nova formulação das DCNs do curso de Medicina. As três versões contam com um instrumento voltado para o professor e outro, espelho, dirigido ao estudante.

Conclusão

Para validar o instrumento sobre competências gerais, direcionado aos professores e graduandos de cursos da área da saúde, foram utilizadas, no que se refere ao seu conteúdo, a busca de referencial teórico, as DCNs e a avaliação de experts; para a validação de critério, o teste t, o teste χ²e a correlação de coeficiente de Pearson; e para validade de constructo, os testes estatísticos de análise fatorial exploratória e confirmatória e o AIC. Para validade de conteúdo, recorreu-se aos mesmos passos descritos para o primeiro objetivo, a fim de garantir a confiabilidade dos instrumentos, e empregaram-se o alfa de Cronbach e o AIC. Após esse processo, foi possível o desenvolvimento de três versões do instrumento, sendo as duas primeiras para utilização de professores e para professores e estudantes em conjunto; e a versão 3 foi apropriada para professores ou estudantes e ainda para professores e estudantes em conjunto, porém ela pode ser utilizada por todos os cursos da área da saúde estudados.

Competência Profissional; Ensino; Avaliação do Ensino; Educação Superior; Estudos de Validação

INTRODUCTION

Health professions, in the world context, have shared challenges regarding the quality of vocational training and the expansion of the practice fields caused as a consequence of changes in work processes. In Brazil, the public policies of the health system are added to these challenges.

In this sense, there are new needs for overcoming and adapting the training models proposed for the health area, a fact that meets the challenges that add to professional competency, defined as the ability to mobilize, articulate and put into action values, knowledge and skills that necessary for the efficient and effective performance of activities required by the nature of the work¹ .

Competencies have been widely discussed in the literature with different concepts. One of the definitions of Competency is the ability to perform and make the opportunity, which includes reconstructive questioning as an innovative basis for the process of development of the capable historical subject. Knowledge is an important tool of competency, but it does not ensure a competent action, since it is the ability and courage to innovate, associated with other cognitive and behavioral skills that shape “the profile of the competent individual”² . This concept has been well accepted and used to instrumentalize individuals with adequate knowledge so that they can reflect on their critical reality³ . The development of competencies is directed at the pursuit of comprehensive health care, contributing to the development of a professional who adds aptitudes for decision-making, communication, leadership, management and continuing education⁴ .

The National Curriculum Guidelines (NCG) in its 2014 issue, bring in its scope, the definition of competency to be assumed by medical education.

In Brazil, the NCG approved between 2001 and 2004 brought to the discussion general competencies for 11 of the 14 courses defined as part of the health area by the National Health Council Resolution N. 287, of 1998. The NCG were defined based on a context of interests and guidelines for the Unified Health System, which focuses on the public, democratic character and the search for universality and comprehensiveness of care⁵ . Therefore, they should be an expression of the commitment that the change in education would have in positioning the University in its social role in the fields of health and education.

They are divided into two major blocks: six General Competencies that include Health Care, Decision-Making, Communication, Leadership, Administration and Management, and Continuing Education, which are common to the Nursing, Pharmacy, Physical Therapy, Speech Therapy, Veterinary Medicine, Nutrition, Dentistry, Psychology, Occupational Therapy and Biomedical Sciences courses; and the General Competencies for the Medical Course, until recently equal to the abovementioned courses, which were reformulated into three blocks in 2014: Health Care, Health Management and Health Education. The Biological Sciences, Social Work and Physical Education courses have a differentiated NCG structure, not including general competencies and the Specific Competencies of each course.

The NCG empowering role in the changes in health undergraduate courses has brought important contributions to the implementation of competency-based curricula, changes in practice, and the use of active methodologies. However, Feuerwerker (2000)⁵ states that these changes are still incipient, emphasizing the existence of inaccuracies in the text of the Guidelines, due to the existence of disputes about future changes that allowed gaps in the adopted guidelines, not favoring the overcoming of the traditional approaches.

Therefore, the assessment of competencies has occupied a prominent place in this discussion. In this context, we increasingly seek to discover, explore and draw positive lessons from evaluations practices, seeking to overcome complex situations associated with teaching that require innovative conceptions of student assessment and maturity⁶ .

Therefore, in search of expanding the intricacies of competencies and their insertions in the health education process, the following objectives of this study stand out: to develop an instrument to support teachers in the evaluation of the General Competencies of undergraduate students from health courses; test the reliability of the instrument with teachers and students from health courses; and validate the instrument on General Competencies.

METHOD

The present was a methodological study, which refers to the investigation of the methods of obtaining, organizing and analyzing the data, discussing the development, validation and evaluation of the research instruments and techniques, aiming at the construction of an instrument that would be reliable, accurate and usable for other researchers to apply⁷ .

For the instrument validation process, we used the model proposed by Raymundo (2009)⁸ and Anastasi (1988)⁹ , which indicate content validity, criterion validity and construct validity.

The content validity , understood as the systematic examination of the instrument’s content, determining its scope in relation to the object to be measured¹⁰ , ¹¹ , ¹² , was followed by the criterion validity , understood as the instrument’s ability to function as a future predictor, of an operationally independent variable called criterion¹³ , because there is no similar validated instrument that aims to evaluate the General Competencies.

To assess the statistical significance of the difference between two sample means for a single dependent variable, we chose the t-Test and also the Chi-square Test, a nonparametric procedure used to test hypotheses about the overall proportion¹⁴ . We also used Pearson’s Linear Correlation Coefficient to measure the linear association between paired quantitative values between sample variables¹⁵ , ¹⁶ .

According to Callegari-Jaques (2003)¹⁷ , the linear correlation coefficient can be evaluated as follows:

And after implementing the described statistical tests, the Akaike Information Criterion (AIC) was assessed, which helps to recognize the importance of a remodeling, in some cases, of the proposed initial model¹⁸ .

The Construct validity was performed to identify and group different measures of some attribute and differentiate them, based on logical and empirical procedures.¹⁴ Factorial Analysis¹⁹ was used for Construct Validity .

For reliability, defined as the degree of consistency between multiple measures of a variable and that indicated the extent to which it can be stably reproduced²⁰ , ²¹ , the Cronbach’s alpha coefficient was used.

The study used the classification proposed by Freitas and Rodrigues (2005)²² , which presents the following scale for analysis of Cronbach’s alpha coefficient:

After assessing Cronbach’s alpha coefficient, the AIC was also used to ensure that the versions proposed in this study were reliable.

For a better visualization, Table 1 shows a validation scheme and the techniques/procedures used in each of the aspects considered in this validation process of the instruments to assess General Competencies.

TABLE 1
Pearson’s coefficient correlation

if 0.00 < ρ < 0.30	there is a poor linear correlation
if 0.30 ≤ ρ < 0.60	there is a moderate linear correlation
if 0.60 ≤ ρ < 0.90	there is a strong linear correlation
if 0.90 ≤ ρ < 1.00	there is a very strong linear correlation

¹⁷

CHART 1
Cronbach’s alpha coefficient internal consistency scale

Reliability	Very Low	Low	Moderate	High	Very High
α value	α ≤ 0.30	0.30 < α ≤ 0.60	0.60 < α ≤ 0.75	0.75 < α ≤ 0.90	α > 0.90

²²

CHART 2
Scheme of validation and reliability/consistency of the assessment instrument for General Competencies of health courses

Characteristic	Type	Procedure/Technique
Validity	Content	– Material search in databases – NCG – Expert evaluation
Validity	Criterion	– t test – Chi-square test – Pearson’s correlation coefficient
Validity	Construct	– Exploratory and confirmatory factorial analysis – AIC
Reliability	Internal Consistency	– Cronbach’s alpha – AIC

Source: Study data.

The data collection site for validation was a Higher Education Institution (HEI) in Curitiba, state of Paraná, which has courses in the health area of Nursing, Biomedical Sciences, Pharmacy, Psychology and Medicine.

A convenience or accidental sampling was used, which selects the available people as study participants¹⁴ .

The inclusion criteria used were:

• for teachers: having the highest workload of the semester in the period of the studied course; have graduated from the same course to which the instrument is being applied.

• for students: being part of the evaluated period;

• The following were excluded from this study:

• All teachers and students who started and did not complete the instrument.

The instrument used was created using the Lime Survey Software, a free environment chosen to apply the online research instrument.

In order to comply with the Guidelines and Regulatory Standards for Research Involving Human Beings, established by Resolution 466/12 of the National Health Council, this study was approved by the Research Ethics Committee under number 826,770.

RESULTS AND DISCUSSION

For the content validity, a survey was performed of the NCGs of all courses in the health area, as well as a research in the Virtual Health Library that includes the LILACS, IBECS, MEDLINE, Cochrane Library and SciELO databases to support the construction of the instruments. After that, a literature review was performed for each of the General Competencies (Health Care, Decision-Making, Communication, Leadership, Administration and Management and Continuing Education); at the time of search for information, for some General Competencies, it was necessary to add the keywords “in the health area”, “and health area” or “General Competencies”, aiming to refine the search, due to the large number of publications related to the topic, totaling 93 evaluated articles.

After the alignment with the NCGs of the 11 courses that have the General Competencies with the search theoretical framework, it was possible to develop Version 1 of the instrument, for the Evaluation of General Competencies in the Health Area, in the option directed to teachers and in the mirror option, adapted to the students (self-evaluation).

Subsequently, a survey of twenty-two experts from the assessed areas was carried out, who were chosen from the Lattes curriculum, publications in the teaching area, those working in the coordination of courses, with a master’s or doctoral degree in the area. Two experts from each course were invited to contribute to the proposed instrument. After the returns, Version 2 of the instrument was developed, maintaining both options (teacher/student).

This version was sent to the survey participants through the Lime Survey Software, after sending an invitation by email, which contained a personal presentation, a brief project report, objectives, deadlines for the feedback and the expected role in the study, as well as a link to answer the instrument. Each teacher participated as a respondent for only one course and period. A total of 51 instruments were retuned by students and 50 instruments by teachers. One instrument from the students that did not have the answers for the teacher’s version was discarded, totaling 50 instruments for each group, all with the corresponding mirror.

The collected data were organized and submitted to statistical analysis carried out with three classifications of different groups: an evaluation called Teacher, which considered only the evaluations made by the teachers; that of the Student, which considered only the students’ evaluations and a third, called the Teachers and Students together, which contemplates the results of the combined Teachers’ and Students’ evaluations.

The results for Criterion and Construct Validity were described and discussed together, following the statistical analyses used for the two validation steps.

The factorial analysis verified the adequacy of the items to the dimensions, as well as measured the degree of effect of each item in this dimension. This analysis proposed the grouping of the six initial dimensions: Health Care, Decision-Making, Communication, Leadership, Administration and Management, and Continuing Education into three dimensions: Health Care, Health Management, and Health Education. The proposal meets the new NCGs approved for Medical education in 2014 that bring exactly the three described dimensions, with no damage to the information to be evaluated.

The criteria used for grouping the information were the chi-square test, the p value of each item and the AIC that allowed identifying the improvements in the model.

As a result, we obtained Version 3 of the instrument, directed to the teacher and the student’s version, which had four questions disregarded, totaling 44 questions (original model with 48), but distributed into three dimensions, instead of six, as it initially did.

The exploratory factorial analysis, both in teacher’s and student’s data, indicated that this is the most appropriate model, and can be applied to teachers and students, alone or together. In all questions, p had a significant value - p < 0.05.

Pearson’s correlation coefficient (r) for this version shows that there is a strong correlation between all dimensions, none of which has negative values.

A strong association is observed between the Health Care and the Health Management dimension: (r = 0.74) in the students’ evaluation and a very strong association (r = 0.92) between the same dimensions in the teacher’s evaluation. When we evaluate teachers and students together, the association remains strong, as well (r = 0.88). The other associations between the dimensions are very close, confirming the association of all dimensions.

Regarding the chi-square test, it showed p <0.001, demonstrating its acceptability for all versions of the instrument.

Regarding reliability , the results were safe and reliable, with the lowest Cronbach’s alpha value of 0.758 in the Health Care Dimension for students, considered a strong value in the evaluation by Freitas and Rodrigues; regarding the other values, they are all strong or very strong, which we desire for all evaluations. Thus, the instruments of this version indicate safety for their use in these groups, separately or together.

TABLE 2
Cronbach’s Alpha Evaluation Table for Students’ Instruments

Dimensions	N	Number of Items	Alpha
Health Care	50	9	0.758
Health Management	50	24	0.9269
Health Education	50	11	0.8021

Source: Study data.

TABLE 3
Cronbach’s Alpha Evaluation Table for Teachers’ Instruments

Dimensions	N	Number of Items	Alpha
Health Care	50	9	0.9024
Health Management	50	24	0.9664
Health Education	50	11	0.9206

Source: Study data.

TABLE 4
Cronbach’s Alpha Evaluation Table for the teachers’ and Students’ instruments assessed together

Dimensions	N	Number of Items	Alpha
Health Care	100	9	0.8555
Health Management	100	24	0.9552
Health Education	100	11	0.9038

Source: Study data.

When comparing Instrument Version 2 with Version 3, it is observed that all values indicated that Version 3 was the most recommended one. When evaluating only the teachers, the AIC value was 4338 for Version 3, while for Version 2 it was 4791; when evaluating only students, the AIC value was 4331 for Version 3; 4871 for Version 2 Model; when evaluated as a whole, the AIC value for Version 3 was 8849 and for version 2, it was 9854; the lower the value in the comparison, the higher the indication.

Thus, the results suggest that the courses that have the six General Competencies could realign them in three dimensions, because statistically, the work showed that the six General Competencies can be directed into three, reducing the number of dimensions without changing their essence.

Based on the NCGs, the competencies are critical to health courses. When assessing the student, it is important to include several aspects that are part of their education, and not only their understanding during the course, which eventually includes the way they interact and their skills²³ .

Thus, it is expected that the students will be able to apply their knowledge in the most diverse situations, no matter how challenging they might be, justifying their competency, whenever necessary. In this sense, one of the main challenges is to evaluate the training of this student through their competencies.

For that purpose, it is imperative the student’s empowerment to be a protagonist in the training process, knowing how and where to seek information and having the capacity to interpret its quality. It is up to the course to encourage these skills of autonomy, criticism and reflection in future doctors.

Since the implementation of NCGs in 2001, the aim has been to change Brazilian education with a focus on what the population really needs. Not understanding the social view of the courses and the lack of research in Education hinder the curriculum development. It is necessary to learn how to connect knowledge to skills, emotions, reasoning and values to consolidate the competency for the good of the patient. This can be developed from early academic immersion in practice focused on person-centered teaching, demonstrating that ethics is much more than the memorization of the code of medical ethics, coming closer to what is advocated by the NCGs²⁴ .

Clearly, the focus of graduation must be to go beyond the technical environment into a fair education.

Competencies can assess quality during the professional training process. According to the French Labor Code, competency development must be part of the teaching of theory, with personal and professional experience ²⁵ .

Within the core of the objectives of changes in the new guidelines is the focus on learning by competency, plus a holistic view of the human being, considering them as a whole, as well as their social context. One way to achieve this is to unite the multiprofessional team with students, teachers and patients. It is essential to apply humanization and ethics in training with the same emphasis as disciplinary research, focusing not only on healing, but also on the care of the individual. For that purpose, it is interesting to reinforce the insertion of a humanistic axis in curricular matrices in Medicine, developing adequate skills for each undergraduate period ²⁶ .

As a result, professionalism has changed due to the social contexts and economic relations of the professions over time. It is important to know what the values, rules and norms of the profession are, and what is the contribution to society in the development of these concepts. Professionalism goes far beyond the Code of Ethics and for that, it is important that both teachers and students be aware of professionalism as the basis of the social contract to legitimize their profession, and it is important to ensure proper training from the correct understanding of this topic and respect its principles, such as always considering first the patient’s well-being and autonomy and the professional’s commitment to the competency, confidence and quality of care with their professional responsibilities ²⁷ .

Returning to the assessed subjects allows the possibility of contextualizing learning, associating theory with practice in health. However, it is common to question why knowledge occurs separately if the objective is to aim for an integrated professional practice. Thus, attention is already given to curriculum integration, which allows new strategies for teaching / learning and academic assessment. For that, multidisciplinarity promotes a more integrated learning ²⁸ .

A model that encompasses professionalism in the curricular matrix of the courses is yet to be observed and, if desired, it should be adapted to the local teaching context in the different realities of education ²⁹ .

The acquisition of knowledge currently highlights technical and scientific skills. Thus, professionalism emerges to renew the teaching-learning process in a fair manner. Although several studies are being conducted to assess competencies, they still show low reliability, and there are few objective instruments that evaluate only some aspects of professionalism. Thus, there is a scarcity in the literature regarding the aspect of professionalism assessment methods and use of instruments, which is the current focus on the topic according to the author, in addition to the importance of incorporating the topic in the curricula. In this type of assessment, it is relevant to include students, health professionals and patients. Since no single instrument addresses all aspects of a person’s performance, it is important to emphasize their expertise, the validity of the instruments, and the appropriate sampling of participants ³⁰ .

CONCLUSIONS

Thus, the study brought, in version 3, a valid model for the assessment of General Competencies, adequate to be used with teachers, students and both, directed to the NCGs of ten health courses that subdivide their General Competencies into six dimensions, in addition to Medicine, now subdivided into three dimensions.

Version 3 is ideal to be used with the eleven health courses (Nursing, Pharmacy, Physical Therapy, Speech Therapy, Medicine, Veterinary Medicine, Nutrition, Dentistry, Psychology, Occupational Therapy and Biomedical Sciences).

This instrument aims to minimize the challenges faced in the evaluation of the General Competencies, considering the difficulty in implementing them at the HEIs.

The implementation of this instrument may contribute for the HEIs to obtain subsidies to evaluate the comprehensive training process of health students based on the General Competencies expected from graduated students. The model allows identifying weaknesses in the process and correcting them before the end of the course and can be used from the beginning to the end of undergraduate school.

The study suggests the consolidation of clear and objective evaluation models for both teacher and student, helping to demystify the evaluation process.

ORGANIZATION OF FUNDING SOURCES

The entire study was developed with the authors’ own financial resources.

REFERENCES

• ¹
  Brasil. Ministério da Educação. Resolução CEB n.4, de 8 de Dezembro de 1999. Institui as Diretrizes Curriculares Nacionais para a Educação Profissional de Nível Técnico. Brasília, 1999.
• ²
  Demo P. Educar pela pesquisa. Campinas: Autores Associados, 2011.
• ³
  Leonello VM, Oliveira MAC. Competencies for educational activities in nursing. Rev. Latinoam de Enfermagem 2008; 16(2)177-183.
• ⁴
  Silva KL, Sena RR. Nursing education: seeking critical-reflexive education and professional competencies. Rev. Latinoam de Enfermagem 2006; 14(5)755-761.
• ⁵
  Feuerwerker LCM. A construção de sujeitos no processo de mudança da formação dos profissionais de saúde. Divulgação em Saúde para Debate 2000; (22)18-24.
• ⁶
  Vasconcellos MMM, Oliveira CC, Berbel NAN. The university teacher and appropriated evaluation practices in higher education: a student’s perspective. Interface. Botucatu: Selected Edition v3, 2007.
• ⁷
  Polit DF, Hungler BP. Fundamentos da pesquisa em enfermagem. Porto Alegre: Artes Médicas, 1995.
• ⁸
  Raymundo VP. Construção e validação de instrumentos: um desafio para a psicolingüística. Letras de Hoje 2009; 44(3)86-93.
• ⁹
  Anastasi A. Psychological testing. New York: MacMillan, 1988.
• ¹⁰
  Anastasi A, Urbina S. Testagem Psicológica. Porto Alegre: Artes Médicas Sul, 2000.
• ¹¹
  Cronbach LJ. Fundamentos da testagem psicológica. Porto Alegre: Artes Médicas, 1996.
• ¹²
  Menezes PR. Validade e confiabilidade das escalas de avaliação em psiquiatria. Revista de Psiquiatria Clínica 1998; 25(5)214-216.
• ¹³
  Fachel JMG, Camey SA. Avaliação Psicométrica: a qualidade das medidas e o entendimento dos dados. In: Cunha JA, org. Psicodiagnóstico. Porto Alegre: Artes Médicas, 2000.
• ¹⁴
  Polit DF, Beck CT, Hungler BP. Fundamentos de Pesquisa em Enfermagem métodos, avaliação e utilização. Porto Alegre: Artmed, 2004.
• ¹⁵
  Triola MF. Introdução à Estatística. Rio de Janeiro: LTC, 2008.
• ¹⁶
  Vieira S. Estatística Básica. São Paulo: Cengage Learning, 2013.
• ¹⁷
  Callegari-Jacques SM. Bioestatística: princípios e aplicações. Porto Alegre: Artmed, 2003.
• ¹⁸
  Bozdogan H. Model selection and akaike’s information criterion (aic): the general theory and its analytical extensions. Psychometrika 1987; 52(3)345-370.
• ¹⁹
  Hair JF, Black WC, Babin BJ. Análise multivariada de dados. Porto Alegre: Bookman, 2009.
• ²⁰
  Vianna HM. Testes em educação. São Paulo: Ibrasa, 1976.
• ²¹
  Schuwirth LWT, Vleuten CPM. General overview of the theories used in assessment. AMEE Guides in Medical Education 57, 2012.
• ²²
  Freitas ALP, Rodrigues SG. A avaliação da confiabilidade de questionário: uma análise utilizando o coeficiente alfa de Cronbach. In: XII Simpósio de Engenharia de Produção Bauru: UNESP; 2005.
• ²³
  Gontijo ED, Alvim C, Megale L. Matriz de Competências Essenciais para a Formação e Avaliação de Desempenho de Estudantes de Medicina. Rev. Brasileira de Educação Médica 2013; 37(4)526-539.
• ²⁴
  Franco RS, Franco CAGS, Portilho EML. O conceito de Competência: uma Análise do Discurso Docente. Rev. Brasileira de Educação Médica 2014; 38(2)173-181.
• ²⁵
  Fragelli TB, Shimizu HE. Competências profissionais em Saúde Pública: conceitos, origens, abordagens e aplicações. Rev. Brasileira de Enfermagem 2012; 60(4)667-674.
• ²⁶
  Franco CAGS, Cuba MR, Franco RS. Currículo de Medicina e as Competências Propostas pelas Diretrizes Curriculares. Rev. Brasileira de Educação Médica 2014; 38(2)221-230.
• ²⁷
  Rego S. O Profissionalismo e a Formação Médica. Rev. Brasileira de Educação Médica 2012; 36(4)446.
• ²⁸
  Iglesias AG, Bollela VR. Integração curricular: um desafio para os cursos de graduação da área da saúde. Rev. Da Faculdade de Medicina de Ribeirão Preto 2015; 48(3)265-272.
• ²⁹
  Birden H, Glass N, Wilson I. Teaching professionalism in medical education: A Best Evidence Medical Education (BEME) systematic review. BEME Guide, Australia 2013; (25)1252-1266.
• ³⁰
  Mendonça ET, Cotta RMM, Lelis VP. Avaliação do profissionalismo em estudantes da área de saúde: uma revisão sistemática. Interface: Comunicação Saúde Educação 2016; 20(58)649-690.

Publication Dates

• Publication in this collection
  13 Jan 2020
• Date of issue
  2019

History

• Received
  15 Apr 2019
• Accepted
  19 June 2019