SciELO - Scientific Electronic Library Online

 
vol.23 issue11Risk and protective factors for suicide attempt in emerging adulthoodComparative study between formal and informal caregivers of older adults author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

  • text new page (beta)
  • English (pdf)
  • Article in xml format
  • How to cite this article
  • SciELO Analytics
  • Curriculum ScienTI
  • Automatic translation

Indicators

Related links

Share


Ciência & Saúde Coletiva

Print version ISSN 1413-8123On-line version ISSN 1678-4561

Ciênc. saúde coletiva vol.23 no.11 Rio de Janeiro Nov. 2018

http://dx.doi.org/10.1590/1413-812320182311.27072015 

FREE THEMES

Collective portfolio: assessment of teaching and learning in health undergraduate courses

Portfolio coletivo: avaliação do ensino e apredizagem em cursos de graduação da saúde

Glauce Dias da Costa1 

Erik Driessen2 

Luciana Saraiva da Silva1 

Aline Aparecida de Oliveira Campos1 

Thiago de Melo Teixeira da Costa3 

Cíntia Pereira Donateli1 

Rosângela Minardi Mitre Cotta1 

1 Departamento de Nutrição , Universidade Federal de Viçosa (UFV). Av. Peter Henry Holfs s/n, Centro. 36570-000 Viçosa MG Brasil . glaucedcosta@gmail.com

2 Department of Educational Development and Research , Faculty of Health Medicine and Life Sciences , Maastricht University . Maastricht Netherlands .

3 Departamento de Administração , UFV . Viçosa MG Brasil .

Abstract

The potential of the reflective collective portfolio as a method of teaching, learning and assessment needs to be emphasized. The objectives is to assess and validate the quality of the collective portfolio as a method of teaching, learning and assessment in the training of health professional. Portfolio quality was analyzed using an inventory of analysis that was adapted and tested for this study. In total, 70 portfolios were analyzed. The Kappa test was used to analyze reproducibility, confirming the level of agreement between the appraisers. Descriptive analysis and the t-test were performed to compare results from the years the portfolios were produced. The constructs (layout and organization, comprehensive, reflective, critical and creative thought) exhibited Cronbach alpha values above 0.7, confirming the high reliability of the tool. The weighted Kappa test revealed an elevated and significant agreement (+90%) between the appraisers. Notably, certain items and constructs exhibited significant differences over the years. All of the constructs were assessed better from 2011 onwards, when the assessment and self-assessment tool was implemented. The success of the portfolio depends on strategies that provide students with greater clarity and forms of developing the method.

Key words: Evaluation; Health education; Medical education

Resumo

O potencial do portfólio coletivo reflexivo como método de ensino, aprendizagem e avaliação precisa ser enfatizado. Os objetivos consistem em avaliar e validar a qualidade do portfólio coletivo como um método de ensino, aprendizagem e avaliação na formação de profissionais de saúde. A qualidade do portfólio foi analisada utilizando um inventário de análise que foi adaptado e testado para este estudo. No total, foram analisados 70 portfólios. O teste Kappa foi utilizado para analisar a reprodutibilidade, confirmando o nível de concordância entre os avaliadores. Análise descritiva e teste t foram realizados para comparar a construção do portfólio nos diferentes anos. Os construtos (layout e organização, compreensão, reflexão, crítica e pensamento criativo) apresentaram valores de alfa de Cronbach acima de 0,7, confirmando a alta confiabilidade da ferramenta. O teste Kappa ponderado revelou uma concordância elevada e significativa (+ 90%) entre os avaliadores. Notavelmente, certos itens e construtos apresentaram diferenças significativas ao longo dos anos. Todas as construções foram avaliadas melhor a partir de 2011, quando a ferramenta de avaliação e autoavaliação foi implementada. O sucesso do portfólio depende de estratégias que proporcionem aos alunos maior clareza e formas de desenvolver o método.

Palavras-Chave: Avaliação; Educação em saúde; Educação médica

Introduction

Portfolios have been defined as a collection of evidence and reflections that students use to demonstrate the results of specific learning and are an effective method of assessing personal and professional development 1 . Thus, the portfolio is a method designed by the students who must identify evidence that allows them to assess their knowledge, skills and attitudes (competence), as well as the learning process itself 2-4 . In this context, the students are considered as agents: “somebody who acts and causes change and whose achievements can be judged by their own values and objectives, regardless of assessments associated with external criteria” 5 .

There are many different types of portfolios 6,7 , including the collective portfolio, which involves the acquisition of competence in a group, encouraging teamwork and training future professionals in problem-solving and decision-making. This version of social reality presupposes that professional training leads to a new rationality. Students must be prepared to provide the most adequate, competent, democratic and effective answers to the enormous challenges that await them in contemporary society, particularly in the area of health. Therefore, the development of skills, in terms of making people and organizations more reflective, competent and efficient, is indispensable in the training of new professionals 8 .

This emphasizes the potential of the reflective portfolio as a method of teaching, learning and assessment, through the development of the following skills: decision-making; knowing how to work as part of a team, including inter-disciplinary teams and occasionally trans-disciplinary teams; communication skills; development of critical, reflective and creative thought; strengthening autonomy in the learning process.

However, in spite of all of the positive points in favor of the reflective portfolio, there is a consensus about the fragility of the assessment and validation of this method in the day-to-day operation of universities. Driessen et al. 9 stated that assessing a portfolio is complex due to its open, dynamic and flexible format. The same authors defended more global assessments with trained appraisers and stated that the richness and complexity of portfolios cannot be captured by analytical assessment criteria and extremely technical checklists can easily trivialize the assessment. From this perspective, it is essential to create portfolio analysis tools that ensure each portfolio is designed for the specific learning context of the student or group in question.

With this fragility of the assessment and validation of portfolios in mind, the aim of the present study was to assess and validate the collective portfolio as a method of teaching, learning and assessment, in terms of professional health training, as well as to present a tool created and tested for this purpose.

Materials and methods

Context

The construction of a collective portfolio is used as a didactic method of teaching, learning and assessment in the discipline of Health Policy for graduate courses in a Brazilian public university. The aim is to use this active method in large groups. The primary purpose of the portfolio is to promote learning associated with health policies, stimulating critical thought and enabling students to decipher reality, acquire autonomy of judgment and share accountability in knowledge building while part of a team. The portfolios are built collectively in groups composed of approximately six students.

At the beginning of the semester, the learning objectives, related to the construction of the portfolio, are designed jointly by teachers and students. In order to guarantee a clear structure with well-defined criteria and guidelines, students are instructed in all of the phases that should be carried out while constructing a portfolio: planning (before the activity); monitoring activities and the monthly assessment process (four assessments per semester); real-time assessments and feedback, with constant dialogue between teachers and students and between the students themselves 3,4,10 .

The teacher who is accompanying the post-graduate students meets the groups, analyzes and discusses the items in the portfolio and promotes debate and questions about the evidence used. It is important to highlight that the assessment tool for the professor and the self-assessment tool for students was created and implemented in 2011, based on the experience of the professor and broad theoretical foundations, having been tested and discussed with students. Its structure is composed of learning objectives and aspects of classification and assessment (needs to improve, progressed adequately, progressed remarkably, exceptional).

Cotta et al. 10 proposed four stages in the construction of a portfolio: 1) Search and conceptual production: the first proposed activity suggests that students search the scientific literature for the concepts, definition and aims of the portfolio (extra-curricular activity) and elaborate an individual and collective concept based on the literature consulted, including a list of the characteristics that a reflective portfolio should contain; 2) My trajectory: memories written at the beginning and end of the semester; 3) Learning with the group: activities designed and conducted in a group using narratives (reports of experiences in different practical scenarios), according to the theses involved; 4) Creative space: free space where the group can express their creativity using cartoons, poems, songs, photos and artwork from written and electronic media, associated with critical thought.

Assessment inventory

An inventory of analysis of the quality of portfolios was created as a result of the studies conducted by Driessen et al. 9 , Gadbury-Amyot et al. 11 and Cotta et al. 3,10 with the aim of assessing portfolios using documentary analysis.

The items (20 questions, Table 1 ) in the inventory were formulated based on previously referenced studies, as well as the learning objectives proposed by the professor responsible for the discipline at the beginning of the process and the assessment and self-assessment criteria contained in the assessment tool.

Table 1 Constructs, items and internal reliability of the inventory of the analysis of portfolio quality. 

Constructs Item Cronbach’s Alpha
Layout and Organization 1 The portfolio is easy to use and the data are easily found 0.8345
2 The stages/sub-divisions (set apart) are easily understood
3 The students worked hard to prepare the layout of the portfolio
4 The students went beyond what was expected of them
Comprehensive Thought 5 The students compared and assessed the different points of view and the content of the portfolio 0.9203
6 There is the formation of hypotheses and ideas. arriving at coherent conclusions with concepts and theories related to health policies
7 The portfolio contains evidence of understanding the role of Health Policy in general and of the SUS system in particular
8 The portfolio contains evidence of investment in training for the citizenship of the students
Reflective Thought 9 There is revision, construction and reconstruction of concepts 0.8616
10 The analysis performed by the students is extensive and transcends a list of facts or situations
11 The students listed the strengths and weaknesses in the process of creating the portfolio
12 The objectives related to proposed reflective thought at the beginning of the discipline were clearly and logically reached
Critical Thought 13 Whenever relevant, statements are accompanied by evidence that supports what is being stated in the portfolio 0.9208
14 The students used different types of sources and evidence in a critical manner
15 There is evidence of the incorporation of the content studied in the discipline
16 There is in-depth and theoretical coherence to the evidence presented in the portfolio
17 The data are useful and significant in the construction of learning and are clearly and coherently related to the aims of the discipline
Creative Thought 18 The students produce and develop ideas 0.7307
19 There is evidence of the transference of the creative process to the content associated with the discipline
20 The students used Information and Communication Technology (ICT)

A Likert scale was created to measure the criteria. The scale was designed with five points and was ordered as follows: clearly not contemplated; partially not contemplated; neutral; partially contemplated; clearly contemplated.

The constructs were formulated based on the theoretical assumptions of Lizarraga 12 adapted by Cotta et al. in 2011 10 and 2013 3 in which cognitive skills were divided as follows: comprehensive thought, which promotes skills such as comparison, classification, analysis, synthesis and the creation of arguments; critical thought, which promotes skills such as the investigation of the reliability of sources, the interpretation of causes, analogical reasoning and deduction; creative thought, which refers to the possibility of generating ideas, establishing associations, producing images, creating metaphors and establishing goals. In addition, theoretical aspects related to studies by Driessen et al. 9 and Gadbury-Amyot et al. 11 were included.

A pilot study was carried out by the research team in order to adjust the formulation of the inventory items and their respective instructions. Ten post-graduate student portfolios were analyzed from the area of health promotion. It is worth noting that the appraiser used the items of the inventory scale for each activity in the portfolio to help analyze the portfolios.

Procedure

In total, 70 portfolios, designed between 2008 and 2013, were analyzed. Three properly trained appraisers performed the analysis. In order to increase inter-appraiser reliability, the appraisers discussed the items of classification of two portfolios, which were excluded from the present study. The analysis was performed blind by all appraisers.

Data analysis

When assessing the measurement properties of an adapted tool, Guillemin et al. 13 proposed an assessment of the reliability and validity of the tool.

The Kappa test, weighted between the three appraisers, was used to analyze reproducibility, confirming the level of agreement between the appraisers. The Kappa values were divided as follows: 0 – poor agreement; from 0 to 0.20 – mild agreement; from 0.21 to 0.40 – considered agreement; from 0.41 to 0.60 – moderate agreement; from 0.61 to 0.80 – substantial agreement; from 0.81 to 1.0 – excellent agreement.

Cronbach’s alpha was used to analyze internal consistency. Index values above 0.70 indicated a good construct. Pearson’s correlation coefficient was used to correlate each of the items in relation to each other and their constructs.

Descriptive analysis (mean and standard deviation) and the t-test were performed with the data to compare results from the years the portfolios were produced (2008 to 2013). The data were analyzed using SPSS 20.0 and Stata 11.0 software.

Ethical approval

The present study received approval from the Ethics Committee of the Universidade Federal de Viçosa (UFV). According to resolution 466/2012 of Brazil’s National Health Council, which regulates studies involving humans, a statement of informed consent was signed by those who agreed to participate in the research. The participants were guaranteed confidentiality and anonymity.

Results

The most significant finding of the present study was the high reliability of the analysis tool (Inventory of collective portfolio quality), as shown in Table 1 . The constructs (layout and organization, comprehensive, reflective, critical and creative thought) exhibited Cronbach alpha values greater than 0.7, which confirmed the high reliability of the tool in terms of analyzing the portfolio.

The weighted Kappa test revealed an elevated and significant agreement between the appraisers, taken two by two. The mean level of agreement was greater than 90%, as shown in Table 2 .

Table 2 Kappa tests and agreement between the appraisers by item of the inventory of analysis of portfolio quality. 

Item Examiners 1 e 2 Examiners 1 e 3 Examiners 2 e 3



Agreement Kappa Agreement Kappa Agreement Kappa
1 94.85% 0.7831 97.06% 0.8742 95.59% 0.8132
2 92.65% 0.7094 94.49% 0.7809 95.96% 0.8384
3 90.44% 0.6918 92.28% 0.7527 92.28% 0.7681
4 95.59% 0.7679 96.69% 0.8251 95.22% 0.7654
5 96.69% 0.8419 97.06% 0.8566 94.49% 0.7333
6 94.49% 0.777 98.16% 0.9295 94.12% 0.7633
7 95.59% 0.8273 95.96% 0.8373 94.49% 0.7843
8 94.12% 0.7486 94.12% 0.7342 92.65% 0.6872
9 93.75% 0.7429 95.22% 0.7927 92.65% 0.7084
10 96.32% 0.858 94.12% 0.7723 93.38% 0.7449
11 97.43% 0.918 95.96% 0.8723 96.32% 0.8852
12 88.60% 0.5107 95.59% 0.8041 87.87% 0.4848
13 95.59% 0.7783 94.49% 0.7236 91.54% 0.6155
14 93.75% 0.713 97.06% 0.8638 92.28% 0.6472
15 94.12% 0.7129 97.06% 0.8486 93.38% 0.6878
16 91.54% 0.6919 96.69% 0.8664 89.71% 0.6317
17 91.91% 0.6065 94.12% 0.7165 89.71% 0.5412
18 94.85% 0.7748 95.22% 0.7889 95.96% 0.8191
19 91.91% 0.725 92.28% 0.7592 89.34% 0.6784
20 98.53% 0.9077 98.16% 0.8863 99.63% 0.9784
Mean Agreement 94.14% 95.59% 93.33%

Therefore, it is worth noting that the analysis tool and the analysis performed by the appraisers were reliable in terms of the assessment conducted, which involved a careful and detailed pre-test, as well as the knowledge and experience of the appraisers in relation to the construction of portfolios.

With regards to the quality of the portfolio, the constructs present in the inventory (layout and organization, comprehensive, reflective, critical and creative thought) can be seen in Table 3 , along with their mean and standard deviation values.

Table 3 Mean and standard deviation per item of each construct of the inventory of analysis of portfolio quality. 

Item 2008 2009 2010 2011 2012 Total






Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD
1 3.35 1.05 4.10 0.70 4.73 0.57 4.83 0.54 4.40 0.63 4.27 0.93
2 3.16 0.92 3.81 0.93 4.50 0.62 4.81 0.40 4.29 0.77 4.11 0.96
3 2.37 0.69 2.81 0.60 4.13 0.79 4.40 0.89 4.52 0.51 3.69 1.15
4 2.76 0.43 3.14 0.36 3.21 0.46 4.07 0.68 3.86 0.84 3.40 0.77
5 2.61 0.49 3.19 0.60 3.00 0.41 3.90 0.53 4.10 0.62 3.33 0.79
6 2.57 0.57 3.38 0.67 3.42 0.61 4.31 0.72 4.19 0.63 3.54 0.92
7 2.63 0.53 3.10 0.44 3.25 0.56 4.24 0.79 4.26 0.86 3.49 0.94
8 2.94 0.42 3.48 0.75 3.58 0.61 4.24 0.88 4.40 0.54 3.72 0.85
9 2.06 0.54 2.86 0.73 2.35 0.48 3.52 0.67 3.55 0.80 2.82 0.90
10 2.47 0.58 3.05 0.80 3.00 0.65 4.10 0.66 3.98 0.81 3.30 0.94
11 2.16 0.86 2.33 0.91 2.60 0.84 3.52 0.99 2.93 1.45 2.72 1.14
12 2.61 0.60 3.24 0.70 3.17 0.60 3.88 0.83 3.88 0.71 3.33 0.85
13 2.65 0.56 2.90 0.44 2.60 0.49 3.62 0.96 3.52 0.71 3.04 0.80
14 2.31 0.47 3.10 0.44 2.92 0.58 3.86 0.65 3.62 0.70 3.12 0.82
15 2.90 0.70 3.19 0.40 3.10 0.37 4.07 0.68 4.24 0.58 3.50 0.80
16 2.18 0.39 2.62 0.80 2.48 0.65 3.67 1.03 3.81 0.67 2.94 0.99
17 3.31 0.51 3.67 0.58 3.69 0.78 4.40 0.63 4.55 0.50 3.92 0.78
18 2.92 0.39 3.19 0.40 3.67 0.72 4.38 0.66 4.40 0.59 3.73 0.84
19 2.18 0.74 3.00 0.84 3.65 0.79 4.12 0.86 4.48 0.51 3.48 1.14
20 1.00 0.00 1.00 0.00 1.00 0.00 1.67 1.51 4.33 1.44 1.82 1.61

SD - Standard deviation.

As can be seen in Table 3 , most of the items exhibited values higher than 3 (neutral and generally contemplated) and these values increased considerably in the years 2011 and 2012. The items with values below 3 were associated with reflective thought when assessing the students capacity to build and rebuild concepts (item 9), thereby demonstrating the student´s difficulty in presenting literature related to the adequate construction of concepts. Another item with values below 3 (also associated with reflective thought) was the presentation of strengths and weaknesses during the construction of the portfolio (item 11). Some of the portfolios omitted this item entirely or only included it in one phase. The critical thought construct (item 16), which deals with deep and theoretical coherence of the evidence in the portfolio, also exhibited a mean value below 3. This was due to the fact that the bibliographical references were not evident or there were few connections between texts.

In the creative thought construct, item 20 exhibited a value below 2 due to the fact that the creation of a group in virtual communities was not obligatory and was meant to facilitate communication and teamwork, as well as helping decision-making and allowing students to make optimal use of the short time they had for meetings. This decreased the final mean value of this construct somewhat. Notably, this item also exhibited the greatest variability between the portfolios analyzed, particularly in the years after 2010, bearing in mind that no portfolio had previously exhibited this characteristic. When analyzing the entire period of the years studied (2008 to 2103), the greatest variability was associated with items 3, 11 and 19.

In spite of the fact that these items exhibited mean values below 3, the vast majority of the items had a final mean value greater than 3. Over the years, a number of items and constructs have reached values higher than 4, including the following: layout and organization (item 1 from 2009 to 2012, 2 and 3 from 2010 to 2012); comprehensive thought (item 5, 6, 7 and 8 in 2011 and 2012); reflective thought (item 10 in 2011); critical thought (item 15 in 2011 and 2012) and creative thought (item 19 and 20 in 2011 and 2012). Figure 1 displays the mean values for the constructs by year for the portfolios assessed.

Figure 1 Evolution of mean values by construct in the analysis of portfolios created by graduate students from health courses in a federal university between 2008 and 2012. 

One notable finding of the present study is the growth of the total mean value of the constructs over time, which indicates appropriation of the process of portfolio construction. The t-test of the mean values, displayed in Table 4 , reveals a significant difference, indicating that all of the constructs were assessed more adequately from 2011 onwards. This in turn indicates the construction of better portfolios, as a result of the acquisition of skills and an improvement in the method. The capacity for comprehensive, reflective, critical and creative thought improved in 2011 and 2012. This improvement may have been due to the use of the assessment and self-assessment tool that was created, implemented and used in the last two years (2011 and 2012). This assessment tool involves learning objectives and assessments for each item, including competence (skills, knowledge and attitudes). This tool enabled a clarification of the method, the assessment criteria and the understanding of the reflective, critical and creative processes.

Table 4 Mean summed scale of each construct before and after 2011 and t-test results for the differences between mean values. 

Constructs Prior to 2011 After 2011 Difference t-test p-value
Layout and organization 14.00 17.60 3.60 9.703 0.000
Comprehensive thought 12.17 16.82 4.65 15.128 0.000
Reflective thought 10.41 14.68 4.27 12.148 0.000
Critical thought 14.30 19.68 5.38 14.895 0.000
Creative thought 7.18 11.69 4.52 15.838 0.000

Costa GD et al.

The portfolios exhibited significantly higher values in the items of reflective and critical thought, which indicates that the students analyzed the views of different studies and scholars to form their own opinions. The analysis of the evidence (cartoons, reports, scientific articles) involves more than a list of facts: it explores themes; it questions the opinions provided; it makes connections and inter-connections between the texts and/or documents of the portfolio; ideas are formed leading to a creative and innovative presentation.

The construct with the greatest final mean value in each year was layout and organization. This demonstrates the ease with which the students organized the portfolio using the required steps, providing supplementary evidence and working in a team to develop and create the portfolio.

Discussion

The findings of the present study indicate improvements in the collective portfolio over the years as a method of teaching, learning and assessment, in terms of the acquisition of important skills on behalf of students. The portfolio contributed to an understanding of the content related to health policy and helped the development of the student’s skills and attitudes, particularly in relation to reflective, critical and creative thought 3,10 .

Gadbury-Amyot et al. 11 suggested that portfolios are a valid and reliable method of assessing the competence of a student. Likewise, Driessen et al. 9 stated that the portfolio is a precise tool when assessing reflective competence.

One of the great authors of contemporary pedagogy, Dewey 14 was cited by Rué 15 , who identified four central aspects that are fundamental to all learning: experience; data for reflection; management and generation of ideas; retention of what was learned. Dewey 14 said that reflection is more than simply an ingredient that is necessary for success. It is in fact an intelligent experience in itself. Thus, reflective training becomes indispensable in the context of professional training. However, it demands conditions of effectiveness that incorporate the desire to reflect, as well as the development of knowledge, using determined activities and materials 15 .

The changes noticed in portfolios over the years demonstrate the importance of strategies created to help students in the process of acquiring skills (assessment and self-assessment tool, definition of learning objectives), as demonstrated by Driessen et al. 9 , and highlight the significance of the capacity and work of students during the reflection process. Based on the increased score of the constructs, it is clear that the items of the constructs were enhanced after the implementation of the assessment and self-assessment tool in 2011. The portfolio has improved as a method of learning and is now more than just an archive folder (with significant characteristics in the layout and organization). It now constitutes a method that focuses the triad of critical-reflective-creative thought and has become an effective method of assessing skills.

With regards to the teaching, learning and assessment processes promoted by the portfolio, the student should receive all of the necessary guidelines at the beginning, including the learning objectives to be achieved, the skills to be developed and the assessment criteria, all of which should be transferred in a transparent and consensual manner. Coll et al. 16 stated that in order for the student to be interested and motivated, they must understand clearly what they are supposed to do and why: “If a student does not know the reason for an exercise or understand what the task involves and what they themselves need, it will be extremely difficult to accomplish an in-depth successful study” 16 . Therefore, the student must act differently, research, create and recreate their knowledge and become autonomous in the learning process. At this moment, the professor abandons his role as a detainer and becomes a guide and epistemological provoker, instructing the students on how to build learning.

Conclusion

The collective portfolio is an important method in skills training. It requires organizational and structural resources that provide the quality necessary to achieve the proposed objectives. In addition to the skills exercised in an individual portfolio, the collective portfolio also promotes teamwork, contributing to the reflection process and the management of conflicts that are inherent in the decision making process. It also contributes to the development of leadership skills and strategies that involve the effective participation of all members of a team in collective tasks and activities. These factors help to prepare the student to work in a multi-professional environment. In addition, feedback related to the collective portfolio is not only provided by the professor, as is the case in individual portfolios, but by the students partners as well, thereby contributing to their process of self-assessment. Further studies are required to ascertain significant differences among the skills promoted by collective and individual portfolios.

The success of the portfolio depends on strategies that provide the students with greater clarity and forms of developing the method, particularly in terms of reflection and critical thought. The analysis of the 70 portfolios revealed that reflective and critical thought have improved in the years 2011 and 2012. This may be the result of the introduction of the assessment and self-assessment tool.

The structure (layout and organization) exhibited significant values, including some greater than 4 (which is the measurement for the scale that is usually used), during all of the years studied. Particular and specific characteristics were presented in the way ideas were organized and presented, which also indicated the development of creativity. In the portfolio assessments highlighted by the literature, there is little evidence in this category. However, the present study presents this aspect as an essential and determining characteristic, within a traditional curricular context, of creating innovative alternatives.

Thus, the inventory of analysis of portfolio quality, which was designed to analyze collective portfolios, was successful in terms of innovating and guiding the analysis of the portfolio and could be adapted to the realities and contexts of different learning processes.

Acknowledgments

The present study was conducted with the support of the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), which is an entity of the Brazilian Government focused on the training of human resources.

References

1. Exley K , Dennick R . Ensenanza em pequeños grupos em Educación Superior . Tutorias, seminários e outros agrupamentos . 2nd ed. Madrid : Narcea ; 2009 . [ Links ]

2. Gonzalez S . Técnicas docentes y sistemas de evaluación em Educación Superior . Madrid : Narcea ; 2010 . [ Links ]

3. Cotta RMM , Costa GD , Mendonça ET . Portfólio reflexivo: uma proposta de ensino e aprendizagem orientada por competências . Cien Saude Colet . 2013 ; 18 ( 6 ): 1847 - 1856 . [ Links ]

4. Cotta RMM , Mendonça ET , Costa GD . Portfólios reflexivos: construindo competências para o trabalho no Sistema Único de Saúde . Rev Panam Salud Pub 2011 ; 30 ( 5 ): 415 - 421 . [ Links ]

5. Sen AK . Desenvolvimento como liberdade . São Paulo : Companhia das letras ; 2000 . [ Links ]

6. Tartwijk JV , Driessen EW . Portfolios for assessment and learning: AMEE Guide no. 45 . Med Teach 2009 ; 31 ( 9 ): 790 - 801 . [ Links ]

7. Tekian A , Yudkowsky R . Assessment portfolios . In: Downing EM , Yudkowsky RBY , editors . Assessment in Health Professions Education . London : Routledge ; 2009 . p. 287 - 304 . [ Links ]

8. Alarcão I . Escola Reflexiva e nova racionalidade . Porto Alegre : Artmed Editora Ltda ; 2001 . [ Links ]

9. Driessen EW , Overeem K , Tartwijk J , Vleuten CPM , Muijtjens AMM . Validity of portfolio assessment: which qualities determine ratings? Med Educ 2006 ; 40 ( 9 ): 862 - 866 . [ Links ]

10. Cotta RMM , Silva LS , Lopes LL , Gomes KO , Cotta FM , Lugarinho R , Mitre SM . Construção de portfólios coletivos em currículos tradicionais: uma proposta inovadora de ensino-aprendizagem . Cien Saude Colet 2012 ; 17 ( 3 ): 787 - 796 . [ Links ]

11. Gadbury-Amyot C , Kim J , Palm R , Mills E , Noble E , Overman P . Validity and reliability of portfólio assessment of competency in a baccalaureate dental hygiene programme . J Dent Educ . 2003 ; 67 ( 9 ): 991 - 1002 . [ Links ]

12. Lizarraga MLSA . Competencias cognitivas em educación superior . Madrid : Narcea Ediciones ; 2010 . [ Links ]

13. Guillemin F , Bombardie C , Beaton D . Cross-cultural adaptation of health related quality of life measures: literature review and proposed guidelines . J Clin Epidemiol 1993 ; 43 ( 12 ): 1417 - 1432 . [ Links ]

14. Dewel J . Democracia y Educacion . Madrid : Morata ; 2004 . [ Links ]

15. Rué J . El Aprendizaje Autónomo em Educación Superior . Madrid : Narcea Ediciones ; 2009 . [ Links ]

16. Coll C , Martin E , Mauri I , Miras , M , Onrubea J , Solé I , Zabala A . O construtivismo em sala de aula . São Paulo : Ática ; 2004 . [ Links ]

Received: November 10, 2015; Revised: November 11, 2016; Accepted: November 13, 2016

Collaborations

GD Costa: draft the article, conception and design; acquisition of data, analysis and interpretation of data and final approval of the version to be published. E Driessen: critical revision of the manuscript to improve intellectual content and final approval of the version to be published. LS Silva, AAO Campos and CP Donateli: acquisition of data and final approval of the version to be published. TMT Costa: analysis and interpretation of data and final approval of the version to be published. RMM Cotta: orientation and supervision of the research, revising it critically and final approval of the version to be published.

Creative Commons License  This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.