Interaction among experience, teaching performance and student's learning at university level

Relação entre experiência e desempenho docente e aprendizagem de universitários

ARTICLES

Interaction among experience, teaching performance and student's learning at university level

Relação entre experiência e desempenho docente e aprendizagem de universitários

Ana Borgobello; Nadia Soledad Peralta; Néstor Daniel Roselli

Universidad Nacional de Rosario, Instituto Rosario de Investigaciones en Ciencias de la Educación, Consejo Nacional de Investigaciones Científicas y Tecnológicas. Bv. 27 de Febrero, 210 bis (2000), Rosario, Santa Fe, Argentina. Correspondência para/Correspondence to: A. BORGOBELLO. E-mail: <[e-mail]>

ABSTRACT

This experimental study approaches characteristics of three university teachers' classes with different levels of experience. According to the literature, experienced teachers more often than not are adaptable to different contexts. However, novice teachers are usually focused on specific class contents to be evaluated. Three teachers were selected, a Professor, and two Assistants (a university level one and an undergraduate). We gave them a source text to teach 10 students each one. After that, students' knowledge was evaluated with a post-test and a self-evaluation. The characteristics of the three classes were evaluated by external judges and the students themselves. Moreover, theme and sequential submission analyses following the source text were made. Results show differences in each of the analyzed aspects according to the literature.

Uniterms: Knowledge level; Learning; Teaching.

RESUMO

O objetivo deste estudo foi relacionar a experiência dos docentes com a apresentação dos conteúdos e a aprendizagem de estudantes no nível universitário. É de se esperar que professores com mais experiência educacional sejam adaptáveis a novos contextos e que não apenas ensinem conteúdos a serem avaliados posteriormente. Metodologicamente, o projeto teve esquema quase-experimental tipo pré-teste/pós-teste. Foram escolhidos três professores de Ciências Exatas: professor experiente, auxiliar com nível superior e auxiliar estudante. Um texto-fonte lhes foi entregue com os conteúdos que deveriam ser ensinados a dez estudantes. As características do ensino foram analisadas quanto ao ajuste das aulas ao texto-fonte e avaliadas por juízes externos à pesquisa e pelos próprios estudantes. A aprendizagem dos conceitos foi avaliada por um pós-teste e uma autoavaliação dos estudantes, ambos de múltipla escolha. Apresentaram-se diferenças nos aspectos analisados nas aulas estudadas. A análise realizada permite evidenciar diferenças nos papéis, possibilitando fortalecer, assim, aspetos diferentes das aprendizagens dos estudantes em futuras aulas que venham a ministrar.

Unitermos: Nível de conhecimento; Ensinamento; Aprendizado.

It is known that at the university teaching, teachers have different levels of experience. Also, it is presumed that these differences in experience in the classroom influence the characteristics of the teaching-learning process developed by them.

In this sense, the socio-cultural approach argues that it is possible to think that these teaching-learning processes are configured in a social space giving them a particular form. According to González Rey (2008), these forms of action are subjective settings and are embedded in the vital plot and social life of people, in other words, in their context. Teachers as Bariani and Pavani (2008) generally do not easily accept that their roles as educators go beyond the mere technicality, not taking into account, for example, the importance of interpersonal relationships in the process.

The literature on performance of experienced university professors and newly qualified teachers is plentiful (Diaz Barriga Arceo & Nuñez Castillo, 2008; Feixas, 2008; Fluckiger, McGlamery & Edick, 2006; García & Montanero, 2004; Longhini & Hartwig, 2007; Mayor Ruiz, 2008; Orland-Barak, 2008). Although studies addressing this issue do not do so with comparisons of experimental designs, most of them do it in a naturalistic manner, with qualitative analysis or through questionnaires. Despite the importance of this methodology, the rigor of controlled comparisons would give the results the possibility to replica.

Zucchermaglio (2003) claims that experienced teachers are generally characterized by having a complex representation of the teaching-learning situation and their skills are more related to the context, allowing them to consider alternatives in their practice. Therefore, they are more flexible, with the possibility of adding to their script contents and examples of concepts. Unlike experienced teachers, newly qualified teachers have as a goal the classroom itself, not having the ability, for instance, of reflecting on both the classroom and the institution as a whole.

The failure in the ability of newly qualified teachers to think about the complex contexts limits the development of classes to only specific contents, having nevertheless a beneficial consequence to students, especially at the time of evaluation. This is because, according to Durán (2004), the newly formed retain sensitivity to the most difficult contents, since these contents were learned recently. Thus, they use simple language that is similar to that used by students.

Bariani and Pavani (2008) found in a study made with questionnaires to teachers and students that most of them had preferences for practical classes and not theoretical ones. These authors believe that it is possible to understand this preference not only because the practical classes have fewer students, but also because there is a greater differentiation of procedures used and teaching is more interactive, whereas theoretical classes would be more traditional.

The subject being studied can also be thought from the perspective of the Instructional Psychology interested in formal education, focusing on the events that take place in the classroom (Genovard & Gotzens, 1990; González Cabanach, Barca Lozano, Escoriza González Nieto & Pineda, 1996; González Calleja, 1996). The interaction is analyzed in a specific teaching-learning context with subjects, students, and teachers, all them also specific. These studies have helped to identify the characteristics of the interaction itself in the classrooms.

Therefore, this study is inserted in two theoretical frameworks: the socio-cultural approach, in the sense of a cultural look in the data set, and the Instructional Psychology, which promotes studies analyzing the here and now of the classroom. The two approaches do not have a univocal epistemological look. However, both result from the need of specifying the approach used in data presented herein. As it was earlier explained, the contextualization of this study is fundamental to the focus which structures this research and, therefore, it will be described in some detail.

The research was situated at the Universidad Nacional de Rosario in Argentina. The Argentine University as a public organization has some peculiar characteristics. A particular relevance is the role the students occupy, both for their participation in political decisions - with equal voice and vote as teachers in the University governing bodies -, and in academic life, participating in the teaching career.

It should be noted that all colleges have students working in the disciplines. A typical composition of teachers in a university discipline in descending order of rank is Titular Professor, Associate and Adjunct (Cathedratic professors), Practical Works Chief (PWC), Assistant with university level (A1) and Auxiliary student (A2). In Argentina they are called Profesor Titular (T), Asociado and Adjunto, Jefe de Trabajos Prácticos (JTP), Auxiliar de Primera and Auxiliar de Segunda, respectively. This is because the university demand is very large, since, in most cases, there is not a selection process for incoming students in undergraduate courses.

Cathedratic professors are responsible for the disciplines and, in general, are those who have more knowledge and trajectory. PWC and A1 work in practical classes or laboratories, the A1 are usually newly qualified teachers. A2 are still students who have successfully passed the subject in which they collaborate and have primarily a learning function as future teachers and/or researchers. A2 are not responsible for the planning and execution of teaching and just do it once a year in the presence of more experienced teachers. The A2 are similar to support teachers, very frequent in Brazilian universities (Natário & Santos, 2010), helping out in the classrooms with a more experienced teacher.

The very characteristic of our university has led us to compare the teaching-learning particularities in three of these teaching positions in an Exact Science career: a Titular Professor (T), an A1 and an A2. The main objective of the study was to relate the teachers' experience with the presentation of contents and students' learning.

Method

The design of this study can be described as a pretest/posttest, and it was quasi-experimental because of the restrictions the own field in study imposes. The independent variable "teachers' roles" is evaluated (T, A1 and A2) on the Dependent Variable I (DVI, teaching) and the Dependent Variable II (DVII, learning).

Dependent Variable I, the characteristics of teaching, has three dimensions. The Dimension 1 (D1) was the adjustment to the subject and the sequence of a source text. The Dimension 2 (D2) was the organization (understood as systematization) and the Dimension 3 (D3) was the quality of knowledge in action.

As for the Dependent Variable II, learning, also has two dimensions: D1 was the recall of information and D2 was the self-assessment the students performed on their own learning process.

Participants

The sample consisted of three teachers and 30 students of a discipline of Exact Sciences. The teachers chosen had different positions in it: T, A1 and A2. The participating students took part as students to the same subject ministered by the teachers.

The study was approved by Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) following the ethical standards of Resolution 2857/06 (Argentina, 2006). Furthermore, it fits to Resolution 6164/202 of the university where the study was conducted (Argentina, 2009). All the people participated freely, the research objectives were explained and anonymity in data processing was guaranteed. Therefore, all the principles and ethical standards required were fulfilled.

Materials

Source text: This text was prepared by the researchers with a topic related to the commonly taught, divided into eight sections or sub-topics that would allow the analysis of the subject and the sequence of this text.

Pretest: A general pre-test with open questions was developed. The objective of this test was to eliminate the people who had prior knowledge of the subject that would be taught.

Posttest: Based on the source text, multiple choice questions and open questions were elaborated.

Questionnaire for students: A first part of the questionnaire allowed the student to evaluate the lesson with a closed question in multiple choice (Very good, Good or Fair). The second part allowed a self-assessment of the learning achieved.

Procedures

The subject for this study was chosen according to the teacher's interest in participating in it. The week prior to the experiment, the teachers received a copy of the source text each, informing them that a series of questions would be made from it and the students would answer them after class. The lessons were developed in parallel, observed and audio-recorded for later detailed analysis. Furthermore, 30 students were selected and they were randomly assigned into three groups. Each teacher taught the same subject to each class.

Firstly, the general pre-test was done. Later, each teacher taught the subject for 30 minutes. After teaching, the post-test and the questionnaire were applied for students.

Data Analysis

a) Dependent Variable I. The characteristics of teaching:

Dimension 1, adjustment to the source text was analyzed as the semantic units spoken by teachers (speech).

The adjustment to the subject of the source text was assessed using two indicators: number of original allusions (not included in source) and omissions to sections of the source text. Each teacher's intervention that was not similar to the text could be classified as directly related or indirectly related to the subject of the lesson. A descriptive-statistical analysis of this indicator was conducted.

The sequential fit analysis was performed from the maximum sequence found that coincided between the source text and semantic units spoken by the lecturer. In other words, the maximum continuous sequence found in each class at any time was registered according to the order of the sections of the source text (for example, if the teacher had the sub-issue seven, followed by 3, 4, 5, 2, the continuum sequence at that moment was three).

Dimension 2, organization (understood as systematization) was measured from the evaluation of the transcripts of audio from five judges external to the experiment. The judges evaluated the organization of the subject and of time. A semantic scale of four options was used.

Dimension 3, the quality of knowledge in action, was also measured from the evaluation of the audio transcripts from the five judges and by the students who were part of the classes. The judges qualified the knowledge quality in action of each class as the clarity and conceptual depth: with a number from "1" to "10" and an evaluative opinion. Students, through the questionnaire, evaluated the class attended as very good, good or fair.

It is important to say that the judges did not know the differences in the experience of teachers and the existence of a source text - to ensure more objectivity in analysis -, and the students only evaluated the class they were part of.

The statistical analysis of the weights of the judges outside the experiment, yielding averages for a comparison between the evaluations were also descriptive.

A qualitative description of the judge's opinions on the teacher's performance in their classrooms was added, bringing a description of the quality of the lessons taught.

b) Dependent Variable II. Learning:

Dimension 1, memory information, was measured objectively by a post-test of content.

Dimension 2 was analyzed through the self-assessment that the students performed on their own learning process.

From the evaluation of the three classes by students and learning outcomes, average and standard deviation for a clearer comparative description was obtained, after bringing up the statistical differences in the calculation of Analysis of Variance (Anova) and t-test for independent samples.

Results

Teaching

Dimension 1: Adjustment to the subject and the sequence of the source text

The adjustment to the subject (Table 1), as it was said, was analyzed following the utterances of teachers. A2 had fewer utterances that did not coincide with the concepts or examples offered in the source text, so her class was more suited to the text. In contrast, T proposed more concepts and examples that were not contained in the written material (46) in comparison with A1 (29) and A2 (6). On the other hand, A2 did not omit any of the eight sections of the source text, whereas T disregarded one and A1 disregarded two.

It can be seen in Table 2 that the A2 class followed a sequential order closer to the source text, as it was the one that showed the greatest continuous sequence of concepts in the same order of the source text (6), compared with T (3) and A1 (2). This analysis demonstrates that its class was the one that most fitted to the source text used in the research.

Dimension 2: Organization-systematization

Regarding the organization-systematization of the subject, the average of the judges' evaluation was similar. However, as expected, the class of T obtained the highest average in the evaluation of this aspect. Here we describe both the statistical analysis and the qualitative description that judges have done about classes (remember that judges could only read the records, they did not know about teachers' differences).

In turn, there was more consensus in the evaluation about the class of T than the A1's and the A2's. The class of T was evaluated four times as organization of subject Moderate and once as Regular, while the A1 was rated Low to High, the same as that of A2. Time organization was better evaluated in class T than both of the others, being qualified only as Moderate or High. Both, A1 and A2, on time organization were not assessed as high by any of the judges.

Dimension 3: Quality of knowledge into action

The conceptual clarity was better evaluated by judges in T. However, it should be noted as important that A2 got better evaluation in this aspect than A1.

While T was criticized by one of the judges, "She is not clear about the measurement error... . Moreover, it does not clarify the concept of... ", this criticism was not sustained by the other judges, that qualified the class as "Very clear and concrete" or "Correct, it describes the concepts required and are included in a practical mark" etc.

As for the class A1, the same judge who highlighted the misconception in class T, noted that in this case "It properly clarifies the measurement error...". Another of the judges said "The subject was referred redrawing some important definitions clearly, without including unnecessary data that would lead to confusion". The other judges criticized some aspects of the lesson related to conceptual clarity: "It is not very clear, the teacher repeats definitions, but does not clarify the doubts of the students...", "I did not find conceptual clarity" or "The class begins and ends in an abrupt way, it is not very clear because sometimes it is stampeded too ".

Regarding the evaluation of conceptual clarity in the A2 class, judges appraised favorably. On the other hand, one of the judges stressed the historical introduction made by A2, unaware that this introduction was part of the text that had been delivered to him for the experiment, "I liked the historical introduction on the concept of... . It conceptualizes the elements very well". Another judge evaluated in an intermediate manner saying that "The concepts that are exposed are clear, though few in number". Finally, the other three judges gave a negative opinion on the conceptual clarity developed in class: "The teacher conveys the subject with little clarity, possibly because he has as unclear to himself what he wants to teach", "The teacher is not clear to convey her knowledge... ", and, "If the teacher has the concepts clear to him, this is not evident in the classroom". "The students' questions to clarify concepts are not answered properly, expressing insecurity" (Table 3).

Regarding the conceptual depth, as expected, T has obtained, on average, the highest scores and A2 has gotten the lowest.

Regarding the qualitative evaluation of judges about conceptual depth, three of them qualified class T in a positive way: "Conceptual depth must be directly related to the future needs of the student. I think depth is quite appropriate to the level"; "Right, the most important related issues are mentioned, denoting a complete understanding of the subject", and, "I think conceptual depth is appropriate to the issues presented". However, the other two judges criticized negatively, even though they have not disqualified teacher's conceptual depth: "I do not doubt the conceptual depth the teacher has. Still, I think the development of the class was oriented more to practical than to concept"; "Moderately well, I consider he handles the concepts, even if he gets confused during the explanation".

Regarding A1's class, there were different opinions from the judges. Two of them were positive: "Despite the dispersion of the subjects, the depth is appropriate" and "It is according to the course level". Another of the judges expressed an opinion fairly critical: "I think it should go deeper taking into account that we are working with students at the university level". A fourth judge found that "It was not conceptually deepened". And finally, a fifth judge issued no opinion.

As for A2, only one of the judges evaluated the class as appropriate: "The deepening of the subject was within the general style, but just clearer and concise ". While all the others criticized negatively: "Regular, I have the uneasy feeling that the importance of the subject is minimized, depth seems to be poor to the level"; "Regular, some concepts... cannot be missing"; "There is no deepening in concepts" and, "It is not very deep in the concepts because it leaves a lot of contents to be developed later on (something that is not necessarily bad a priori)".

As for the evaluation of students (Table 4), T was the best rated class.

In short, it could be said that the class taught by the most experienced teacher, T, was the highest evaluation. These results coincide with what was expected, taking into account that T is the one who has more experience teaching.

Learning

The post-test used to measure the learning achieved had six questions: four multiple choice, one with a relationship between words and their definitions and one open. Students could get a score that ranged from 1 to 100. In the class of T they have gotten an average score of 70.45, in the class of A1 they have gotten 59 and of A2 they have gotten 81.5 (Table 5).

The Analysis of Variance test provides statistically significant differences on the results at p=0.001 for all the three classes. The t Test for two independent samples shows statistically significant differences between the learning outcomes achieved by students of class T and the A2 (p≤0.031). The same difference between the classes of A1 and T was not registered.

Additionally the students' opinion about the learning achieved was measured. Contrary to the objective results obtained in the post-test, students of class A1 realized that they have learned more (59.38) and A2 (43.75) less.

Discussion

The results of this study show that there are differences between teacher's roles teaching in relation to their experience. It is important to remember that the study was on the characteristics of the teacher's classes with different experience and because of a methodological option, it was necessary to take them all to a similar situation.

Thus, possible intervenient variables were controlled, such as the number of students who work with them every day. We believe that this methodological decision does not change the context of the classroom sufficiently to change the subjective settings mentioned by González Rey (2008).

Therefore, the results describe in the studied variables some of the differential characteristics of the teacher's roles in relation to their experience. This experience implies especially having developed different roles in the classroom.

In regard to teaching (D1), the analysis of the adjustment to the subject and the sequence of the source text (D1) show that A2 is the one that submits better to the text on both of the indicators. The more experienced teacher presents lessons with more conceptual opening, probably due to their knowledge of the worked subject and to their own role in the discipline. Therefore, teachers with little experience would fit more to a predetermined screenplay (script) for education, consistent with the description of Zucchermaglio (2003).

Regarding the organization of the subject and time (D2) and the clarity and conceptual depth (D3), the judges qualified, on average, the class of T better than the others. It is important to remember, as it was stated earlier, that the judges were unaware that teachers had different levels of experience and roles at the university. This analysis of the results would be showing that through academic speech it is possible to see differences in the experience and roles of teachers, axis of this work.

In agreement with these results, students who participated in the class T rated it better than those who participated and qualified the other two classes. This result is different from the one found by Bariani and Pavani (2008) where both students and teachers have preferences for the theoretical classes to the practical ones.

On the learning achieved (DVII), students achieved better results on post-tests of class A2 than the other two (statistically significant differences showed up). These results confirm the alleged prior, that is, the A2 in their explanations would be based on the difficult points to understand and with a close social language and cognitively (Durán, 2004).

Apparently, the characteristics of the guide carried by a pair with more experience would assure immediate learning, similar to that measured in the post-test. It is possible that the knowledge guaranteed by the most experienced teacher establishes itself more in the long-term and be more complex in its composition, being very difficult to evaluate it in a post-test. As mentioned in the beginning of this section, these were the constraints that an experimental design of this type presents, where the evaluation involves only immediate and specific learning. With another format that included general or meta-cognitive questions, it would have been possible to assess the type of learning guaranteed by the experienced.

As previously written, the distinct types of methodology lead to important results, but focusing on differentiated characteristics of the phenomenon studied. Therefore, different methodological approaches have distinct limits in the conclusions on how far we can go with them.

Final Considerations

The design presented here allowed us to observe features in the context of the classroom with teachers in different roles at the university in a quasi-experimental controlled situation. Although the results cannot be generalized, they allow us to reflect on the characteristics of the education of young and experienced teachers at the university and, therefore, generate reflections on themselves to improve their practices in classrooms reaching the consequences that teaching has on learning.

However, the authors of this paper believe that this study should be replicated in other disciplines of the university for possible comparisons. It is important to highlight the relevance of this type of research to reflect on the teaching-learning practices at this particular level of the educational system, not forgetting the inner characteristics of the studied context.

Received on: 31/3/2010

Final version on: 24/2/2012

Approved on: 3/4/2012

Acknowledgment: We would like to thank Daniel Paula for the English version, Natalia Ricciardi and Rafaella Improta for the review of the version in Portuguese and Martín Dominino for the collaboration in the research.

• Argentina. (2006). Ministerio da Educación, Ciencia e Tecnología. Resolución 2857/06. Lineamientos para el comportamiento ético en las Ciencias Sociales y Humanidades Buenos Aires: Consejo Nacional de Investigaciones Científicas y Técnicas.
• Argentina. (2009). Universidad Federal do Rosário. Resolución Despacho 6164/202 C.S. 855/2009. Normas éticas en la investigación científica para la Universidad Nacional de Rosario Rosario: Autor.
• Bariani, I. C., & Pavani, R. (2008). Sala de aula na universidade: espaço de relações interpessoais e participação acadêmica. Estudos de Psicologia (Campinas), 25(1), 67-75. doi: 10.1590150103-166x2008000100007.
• Díaz Barriga Arceo, F., & Nuñez Castillo, P. (2008). Formación y evaluación de profesores recién formados: problemáticas y retos. Reencuentro, 53(1), 49-61.
• Durán, D. (2004). El aprendizaje entre iguales: movilizar la capacidad mediadora del alumnado para sacar provecho de la diversidad. Aula de Innovación Educativa, 132(1), 75-76.
• Feixas, M. (2008). El profesorado novel: estudio de su problemática en la Universitat Autónoma de Barcelona. Revista de Docencia Universitaria, 2(1), 13-29.
• Fluckiger, J., McGlamery, S., & Edick, N. (2006). Mentoring teachers' stories: Caring mentors help novice teachers stick with teaching and develop expertise. Delta Kappa Gamma Bulletin, 72(3), 8-13.
• García, G., & Montanero, M. (2004). Comunicación verbal y actividad conjunta en el aula de apoyo: un análisis comparativo entre profesores expertos y principiantes. Revista Española de Pedagogía, 62(229), 541-560.
• Genovard, C., & Gotzens, C. (1990). Psicología de la Instrucción Madrid: Santillana Aula XXI.
• González Cabanach, R., Barca Lozano, A., Escoriza González Nieto, J., & Pineda, J. A. (Eds.). (1996). Psicología de la instrucción: aspectos históricos, explicativos y metodológicos (Vol. 1). Barcelona: EUB.
• González Calleja, F. (1996). Metodología de investigación en psicología de la instrucción. In R. González Cabanach, A. Barca Lozano, J. Escoriza Nieto & J. A. González Pineda (Eds.), Psicología de la instrucción: aspectos históricos, explicativos y metodológicos (Vol. 1). Barcelona: EUB.
• González Rey, F. (2008). Subjetividad social, sujeto y representaciones sociales. Diversitas, 4(2), 225-243.
• Longhini, M. D., & Hartwig, D. R. (2007). A interação entre os conhecimentos de um professor atuante e de um aspirante como subsídio para a aprendizagem da docência. Ciência & Educação, 13(3), 435-451.
• Mayor Ruiz, C. (2008). El desafío de los profesores principiantes universitarios ante su formación. In C. Marcelo (Coord.), Profesores principiantes e inserción en la docencia (pp.177-210). Madrid: Octaedro.
• Natário, E. G., & Santos, A. A. (2010). Programa de monitores para o ensino superior. Estudos de Psicologia (Campinas), 27(3), 355-364. doi: 10.1590/S0103-166X2010000300007.
• Orland-Barak, L. (2008). El impacto de las experiencias prácticas en los profesores principiantes: mentorazgo para formular preguntas diferentes. In C. Marcelo (Coord.), Profesores principiantes e inserción en la docencia (pp.155-176). Madrid: Octaedro.
• Zucchermaglio, C. (2003). La práctica de la enseñanza y el aprendizaje de los que enseñan. In C. Pontecorvo (Coord.), Manual de psicología de la educación (pp.295-308). Madrid: Editorial Popular.

Publication Dates

• Publication in this collection
  02 Sept 2013
• Date of issue
  June 2013

History

• Received
  31 Mar 2010
• Accepted
  03 Apr 2012
• Reviewed
  24 Feb 2012