Acessibilidade / Reportar erro

THE IMPACT OF MULTIMODAL INSTRUCTION ON THE ACQUISITION OF VOCABULARY

O IMPACTO DA INSTRUÇÃO MULTIMODAL NA AQUISIÇÃO DE VOCABULÁRIO

ABSTRACT

The use of inadequate and outdated practices, such as memorizing lists of decontextualized words and the excessive use of translation when teaching vocabulary, represent a prevailing problem in English language teaching. This study aims to determine whether the use of multimodality significantly improves vocabulary acquisition among school students. To determine how the use of multimodality affects vocabulary acquisition, this study was conducted in four groups, 8th, 9th, 10th and 11th graders, from two semi-public schools. The action research intervention was divided into five sessions, in which thirty words were taught in total, dividing them into six words per lesson. To measure the progress, students were given a pre-test before the intervention, and a post-test after being exposed to multimodal input. In addition, a learning style test (VARK) was administered to help them adapt to the material and activities to be applied. The results showed that there is statistically significance between the pre-test and post-test, so it is accepted the research hypothesis Does the use of multimodality in the classroom improve the acquisition of vocabulary in 8th, 9th, 10th and 11th graders from semi-public schools?

Keywords:
multimodality; vocabulary acquisition; primary and secondary education.

RESUMO

O uso de práticas inadequadas e desatualizadas, tais como a memorização de listas de palavras descontextualizadas e o uso desmedido da tradução de conteúdos no ensino de vocabulário, representam um problema importante. Neste estudo se pretende demonstrar que o uso da multimodalidade melhora significativamente a aquisição de vocabulário nos estudantes. Para determinar como o uso da multimodalidade afeta a aquisição de vocabulário, este estudo foi realizado em quatro grupos de dois estabelecimentos educacionais particulares subvencionados pertencentes a 8º ano básico, 1º, 2º e 3º do ensino secundário. A intervenção foi dividida em cinco seções, nas quais se apresentaram trinta palavras em total, dividindo-as em seis por curso. Para medir o avanço, foi aplicado aos estudantes um pré teste antes das intervenções, e um pós teste depois de ter sido expostos ao tratamento multimodal. Ademais, foi aplicado um teste de estilos de aprendizagem chamado VARK, para adequar o material e aas atividades que se realizariam. Os resultados obtidos projetaram que existe significância estatística entre o pré teste e o pós teste, razão pela qual se aceita a hipótese de investigação O uso de modalidade na sala de aula melhora a aquisição de vocabulário em estudantes de 8º ano básico, 1º, 2º e 3º do ensino secundário pertencentes a colégios particulares subvencionados.

Palavras-chave:
multimodalidade; aquisição de vocabulário; ensino primário e secundário.

INTRODUCTION

The study of vocabulary has, for a long time, been set aside when studying how to teach a language, as stated by O'Dell (1997, p. 258):"the topic of vocabulary and lexis are absent from major books on the syllabus and theory of language teaching throughout the 1970s and 1980s". The cause of this issue is the importance of grammar over vocabulary; however, according to Thornbury (2002THORNBURY, S. (2002). How to teach vocabulary. Harlow: Longman.) the study of grammar does not help to improve English language, as the learning of words and expressions would do. Techniques, such as the use of bilingual dictionaries and the memorization of long isolated word lists are still being used in today's classrooms. As Mehring (2005MEHRING, J. (2005). Developing vocabulary in second language acquisition: From Theories to the Classroom. Available at: http://www.hpu.edu/CHSS/LangLing/TESOL/ProfessionalDevelopment/200680TWPfall06/03Mehring.pdf. Accessed: March 5, 2015.
http://www.hpu.edu/CHSS/LangLing/TESOL/P...
, p. 4) suggests "learning new words from a word list is much different from learning them in the context of a sentence". Therefore, it is important that teachers realize that learning a new language is not just a memorization process or acquiring new structures; it is the connection between the words and their reality.

These reasons are the main motivation of this research about one of the newest methods that is being applied to teach English: multimodality. This method began developing in the late 20th century, and has its roots in the field of literacy and the development of reading and writing skills. Its foundation lies mainly in presenting the information through more than one sensory mode using varied materials, e.g. visual (images, videos), audio (music, sounds), gestural (movements, facial expressions), and so on. Modes are different ways to communicate, create meaning and finally understand the world. This study is intended to determine the influence of multimodality on vocabulary acquisition among 8th, 9th, 10th and 11th graders, from two semi-public schools in Chile. This type of schools are funded both by the state and by parents through a monthly fee and they are one of the educational realities in Chilean schooling system. This paper is part of the FONDECYT 1151476 project, entitled "Las dimensiones cognitivas, afectivas y sociales del proceso de planificación de aula y su relación con los desempeños pedagógicos en estudiantes de práctica profesional y profesores nóveles de pedagogía en inglés".

1. REVIEW OF THE LITERATURE

1.1 Vocabulary acquisition

Words are the basis of communication. This is exemplified by Hunt and Beglar (2005HUNT, A. AND BEGLAR, D. (2005). A framework for developing EFL reading vocabulary. Reading in a Foreign Language. 17 (1), pp.24-44., p. 24), who argue that "the heart of language comprehension and use is the lexicon". The main reason of learning words is that thoughts and ideas can be expressed without using grammar rules as a central tool. Nonetheless, to express our thoughts without words would be really difficult. For this reason, vocabulary is highly important to any language, because without enough words used appropriately and in context, people would not be able to understand what others express or state what there is in their own minds.

As Wilkins (1972WILKINS, D. (1972). Linguistics in language teaching. London: Arnold. Word [Def. 1]. (n.d.). Oxford Dictionary Online. In Oxford Dictionary. Available at: http://www.oxforddictionaries.com/us/definition/americanenglish/word?SearchDict Code=all. Assessed: December, 15, 2014.
http://www.oxforddictionaries.com/us/def...
, pp. 111-112) says "while without grammar very little can be conveyed, without vocabulary nothing can be conveyed". A substantial amount of words used in a communicative situation allows people to better understand a message, and to better convey ideas while trying to express what they think or feel. This viewpoint is supported by Krashen (1982KRASHEN, S. (1982). Practice in second language acquisition. Oxford: Pergamon Press., p. 80) who states that "while knowledge of vocabulary may not be sufficient for understanding all messages, there is little doubt that an increased vocabulary helps the acquirer understand more of what is heard or read". Certainly, such is the importance of lexis that some authors, like Lewis (2000LEWIS, M. (ED.). (2000). Teaching collocation. Hove: Language Teaching Publications., p. 8) have gone even further in arguing that "the single most important task facing language learners is acquiring a sufficient large vocabulary".

To begin to use a word, students have to go through different processes. As Bloom (2000BLOOM, L. (2000). The intentionality model of word learning: How to learn a word, any word. In: Golinkoff, R., Hirsh-Pasek, K., Bloom,L. (ed), Becoming a word learner : A debate on lexical acquisition. 1st ed. USA: Oxford University Press., p. 27) states "using a word requires that the child both constructs and holds in mind an intentional-state representation, retrieve the word from memory for expressing an element or elements in the intentional state, and articulate the word in an expression" Therefore, even though it is thought that the process of producing a word is relatively easy, it needs to go through different complex cognitive processes (Hill and Miller, 2013HILL, J. AND MILLER, K. (2013). About classroom instruction that works with English language learners. 2nd edition. Denver: McRel.).

Another important factor in the learning process is context. Yongqi Gu (2003, p. 14) affirms that "learning new words through context is only one step students may use, and that students should think metacognitively and learn new words within the context of where they appear".

For second language acquisition of vocabulary, Krashen (1982KRASHEN, S. (1982). Practice in second language acquisition. Oxford: Pergamon Press.) explains that there are five stages that a learner goes through while learning a new language. Those stages are described in Figure 1.

Figure 1:
Stages of second language acquisition (Hill and Miller, 2013HILL, J. AND MILLER, K. (2013). About classroom instruction that works with English language learners. 2nd edition. Denver: McRel., p. 12)

1.2. Word knowledge

There is a great deal of discussion in relation to which is the most appropriate definition of a word. Carter (1998CARTER, R. (1998). Vocabulary: Applied linguistic perspectives. London, GBR: Routledge.) proposed three possible definitions according to different categorizations: orthographic, lexical and phonetic. The orthographic definition of a word states that "a word is any sequence of letters bounded on either side by a space or punctuation mark" (p. 4). Nevertheless, this statement is limited to written language; it does not include spoken discourse. A second definition explains "a word as the minimum meaningful unit of language" (p. 4), which allows users to differentiate the multiple meanings of some words. However, there are some single units of meaning that are composed of more than one word, such as 'taxi driver' or 'school teacher'. The last definition is closely related to the phonetic field it states that "a word will not have more than one stressed syllable" (p. 4).

As the main topic of this study is word knowledge, it is important to define what it means to know a word. According to Meara (1996aMEARA, P. (1996a). The vocabulary knowledge framework. [ONLINE] Available at: http://www.lognostics.co.uk/vlibrary/meara1996c.pdf. Accessed: January 14, 2015.
http://www.lognostics.co.uk/vlibrary/mea...
, p. 2) knowing a word "means knowing many of the different meanings associated with the word". A learner should not only be able to identify the word and its meaning, but also other meanings that the word may have according to its function in the part of speech. For example, favorite can be used as a noun or an adjective, but it has a difference in meaning.

Learning vocabulary requires effort, practice and time. Nevertheless, it is of great importance to teach new words in context and in different ways for students to develop other aspects of their global knowledge. An example of this is what Mehring (2005MEHRING, J. (2005). Developing vocabulary in second language acquisition: From Theories to the Classroom. Available at: http://www.hpu.edu/CHSS/LangLing/TESOL/ProfessionalDevelopment/200680TWPfall06/03Mehring.pdf. Accessed: March 5, 2015.
http://www.hpu.edu/CHSS/LangLing/TESOL/P...
) states:

Learning vocabulary is an ongoing process, which requires systematic repetition to help students learn, especially low context vocabulary. Students can retain the vocabulary they find useful and relevant to their subject matter by learning vocabulary through context, cooperative learning, and using technology (p. 3).

In general terms, according to Graves et al. (2012GRAVES, M., AUGUST, D. AND MANCILLA-MARTINEZ, J. (2012). Teaching vocabulary to English language learners. TESOL International Association/Teachers College Press.) there are four kinds of vocabulary, which are classified as the following: "words we understand when we hear them (receptive/oral), words we can read (receptive/written), words we use in our speech (productive/oral), and words we use in our writing (productive/written)" (pp. 10-11). For example, on one hand, when children enter school, their oral vocabulary is larger than their reading vocabulary. On the other hand, literate adults have a more developed body of reading vocabulary than the oral one. This does not mean that one type of vocabulary is more important than the other, but that they can develop at different stages.

Regarding the receptive and productive knowledge of words, Nation (2001NATION, P. (2001). Learning vocabulary in another language.. Cambridge: Cambridge University Press ) describes which are the aspects and components that are needed to fully know a word. Those characteristics are exemplified in Figure 2.

Figure 2:
Description of what is involved in knowing a word.

1.3 Lexical competence

There is no agreement among researchers in relation to the definition of lexical competence. Although, Goy (2002GOY, A. (2002). Grounding meaning in visual knowledge In: Coventry, K. & Olivier P., Spatial language: Cognitive and computational perspectives. Dordrecht: Kluwer Academic Publishers.) provides a consistent explanation of what the term could entail, stating that "the way we use, perceive and conceptualize objects is part of a kind of knowledge that not only belongs to our lexical competence, but is precisely what allows us to know the meanings of words and to use them correctly" (p. 122). Among different researchers, there are two main forms of approaching the concept. The first one treats lexical competence as separate traits, as Nation (2001NATION, P. (2001). Learning vocabulary in another language.. Cambridge: Cambridge University Press ) does, describing and classifying every single aspect of the lexical competence, including the derivations of a word and the degree of encounter of a word in speech or print. The second approach to lexical competence simplifies the description of it. Meara (1996b) argues that the separate traits models are not optimal to develop the idea of lexical competence, instead "lexical competence might be described in terms of a very small number of easily measurable dimensions" (p.37). Henriksen (1999HENRIKSEN, B. (1999). Three dimensions of vocabulary development. Studies in second language acquisition 21: 303-317.) described lexical competence in a more balanced way, between the global and the separate traits view. According to her three-dimension model, we have partial-precise knowledge, depth of knowledge, and receptive-productive knowledge, as seen below.

1.3.1. Partial-precise knowledge dimension

This dimension was originally proposed by Meara (1996b) and it is directly related with vocabulary size or breadth. In other words, it could be understood as the amount of words that a person knows.

1.3.2. Depth of knowledge dimension

This dimension could be understood as how well you know a word. Read (1993READ, J. (1993). The development of a new measure of L2 vocabulary knowledge. Language Testing, 10, pp.355-371., p. 357) defines the word depth as "the quality of the learner's vocabulary knowledge". Different researchers have pointed out the difficulty in defining the depth of knowledge, due to the different types that constitute the full expertise of a word. The full understanding of a word does not only mean to know a translation but its synonyms, antonyms, and hyponyms among other relations.

1.3.3. Receptive-productive knowledge dimension

This dimension deals with the ability to understand a word in comprehension and then use it in production. Different researchers (Nakata, 2006NAKATA, T. (2006). Implementing optimal spaced learning for English vocabulary learning: Towards improvement of the low-first method derived from the reactivation theory. The JALT CALL Journal, 2(2); Nation, 2001NATION, P. (2001). Learning vocabulary in another language.. Cambridge: Cambridge University Press ) accept that there is a difference between receptive and productive vocabulary. In relation to that, there are significant differences between how well a person can master receptive and productive vocabulary.

1.4. Multimodality

Multimodality is an interdisciplinary approach, which reacted to the impact of information and communication technologies (ICT). These innovations have shaped the way communication has evolved and, as a result, have set the ground for doing research studies on the functions of language. For example, Unsworth (2008UNSWORTH, L. (2008). Multimodal semiotics: Functional analysis in contexts of Education. London, GBR: Continuum International Publishing.) considers that language is influenced by Systemic Functional Linguistics (SFL), in which "the structures of language have evolved (and continue to evolve) as a result of the meaning making functions they serve within the social system or culture in which they are used" (p.1). Considering the previous point, it is stated that multimodality was first encountered by classical rhetoricians who realized that when communicating, different factors, such as voice, tone, facial expressions and gestures, were relevant for the message to be conveyed (Wysocki, 2002WYSOCKI, A. (2002). Teaching writing with computers: An introduction (3rd ed.). Boston: Houghton-Mifflin. p. 182-201.).

Later, during the mid-20th century, with the spreading of new technologies such as photography, video and audiotape, there was an increasing predisposition about using these technologies in the classroom. As research suggests, different modes of representation (visual, audio or tactile), individuals' interaction (gestural, physical or visual), and communication, have changed the way information is perceived, attended to and understood.

Moreover, at the end of the 20th century with the increasing research studies in the cognitive learning field, it became indisputable that different ways of representation or action influence human attention, interaction, perception, interpretation, and construction of meanings. In addition, it has been demonstrated that using different modes improves the student's perception on his or her own learning process; learners are more conscious and active towards the language. This idea is explained by neuroscience as stated in Sankey, Birch and Gardiner (2010SANKEY, M., BIRCH, D. AND GARDINER, M. (2010). Engaging students through multimodal learning environments: The journey continues. In: C.H. Steel, M.J. Keppell, P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010. pp. 852-863., p. 853) "significant increases in learning can be accomplished through the informed use of visual and verbal multimodal learning".

It is essential to mention that not only multimodality is important for the student's learning process, but also to stimulate learners with a motivating environment. In other words, as stated by Sankey, Birch and Gardiner (2010SANKEY, M., BIRCH, D. AND GARDINER, M. (2010). Engaging students through multimodal learning environments: The journey continues. In: C.H. Steel, M.J. Keppell, P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010. pp. 852-863., p. 853) "students may feel more comfortable and perform better when learning in environments that cater for their predominant learning style". Chen and Fu (2003CHEN, G. AND FU, X. (2003). "Effects of multimodal information on learning performance and judgment of learning". Journal of Educational Computing Research, 29(3), 349-362., p. 359) highlight that "multimodal information presentation makes people feel that it is easy to learn and they can maintain their attention span, which will benefit the learning process and increase the learning performance". Therefore, multimodal resources are not only important to engage students' attention but also to improve their performance (POURHOSSEIN, NIZAN AND MASOUMEH, 2011POURHOSSEIN, A., NIZAM, H. AND MASOUMEH, S. (2011). The effect of multimodal learning models on language teaching and learning. Theory and Practice in Language Studies, Vol. 1, No. 10, pp. 1321-1327.).

There are different research studies that support the fact that multimodal lessons and the use of multimodality could be a vital element during the learning process. Mayer declares "that students learn more deeply from a combination of words and pictures than from words alone; known as the, multimedia effect" (2003:307). Fadel agrees with Mayer's idea, when he affirms that "students (who get) engaged in learning that incorporates multimodal designs, on average, outperform students who learn using traditional approaches with single modes" (2008:13).Consequently, one of the major benefits of multimodality, as identified by Picciano is that it "allows students to experience learning in ways in which they are most comfortable, while challenging them to experience and learn in other ways as well" (2009:13). Considering that only one mode can influence in such a positive way the learning process, a combination of them in a multimodal classroom could definitively help students in creating meaningful and long-lasting knowledge.

1.4.1. Multimodality in education

Multimodality has been developed over the past few years to systematically address much-debated questions about changes in society, for example, in relation to new media and technology. Globalization and new technologies, such as computers, mobile phones, and mostly, Internet access, have changed the way in which students acquire, perceive and transform knowledge, and how they stand and behave in the classroom. It proposes a new challenge for teachers to engage their attention and to keep their practices up-to-date. A related idea is exemplified by Kosnik, Roswell and Williamson who state that "children, adolescents, adults and seniors today engage daily in new communicative practices, with new tools, using diverse and multiple modes, and across global landscapes" (2013:36). With the beginning of the technological era in the 21st century, multimodality gained importance in the educational area, not only to engage students and motivate them, but to develop a new form of literacy in children. This can be seen in the research carried out by Hasset and Curwood (2009) where they propose multimodal instruction in elementary classrooms. Yelland, Lee and O'Rourke state "multimodality affords educators an opportunity to draw on students' funds of knowledge" (2008:47). In addition, the new modes that technology is offering invite us to think that new cognitions are needed on the part of those individuals who are processing information and constructing knowledge in a non-traditional manner and who have been doing so for decades now. Related to these new modes, Bezemer and Jewitt (2010BEZEMER, J. AND JEWITT, C. (2010). Multimodal analysis: Key issues. In: L. Litosseliti(ed), Research methods in Linguistics. London: Continuum. pp.180-197.) state that until recently, the dominance of image over word was a feature of texts, on screen and off screen: there are more images on screen and images are increasingly given a designed prominence over the written elements. These authors lead us to reflect upon the necessity to update the teaching practices that should be used in classroom nowadays.

1.4.2. Modes

When learning any topic, it is fundamental to have in mind that there are different ways to create meaning, to understand the world and perform actively in our own learning process; those different ways to communicate are called modes. According to Bezemer and Jewitt: "Modes are an organized set of resources for making meaning" (2010BEZEMER, J. AND JEWITT, C. (2010). Multimodal analysis: Key issues. In: L. Litosseliti(ed), Research methods in Linguistics. London: Continuum. pp.180-197.:183); for example a gesture, an image, or a piece of writing. Furthermore, these authors also describe what requirement a mode needs; "in order for something to be a mode there needs to be a shared cultural sense within a community of a set of resources and how these can be organized to realize meaning" (Ibid).

In addition, the socio-linguistic approach states that the analysis of the resources of a mode may change the meaning of communication, Bezemer and Jewitt state that "modes can merely expand, exemplify or modify the meanings of speech or writing, by analyzing the lexis, intonation, rhythm and tone, hesitations and restarts, and non-verbal communication" (2010:181). Furthermore, this idea was supported by Gumperz when he claimed that "any verbal sign which when processed in co-occurrence with symbolic grammatical and lexical signs serves to construct the contextual ground for situated interpretation" (1999:461). It is relevant to teach a learner how to analyze all the elements that are involved in the construction of a mode and the meaning that entails.

One of the benefits of using different modes during the process of learning is that they "encourage students to develop a more versatile approach to their learning" (Sankey, Birch and Gardiner, 2010SANKEY, M., BIRCH, D. AND GARDINER, M. (2010). Engaging students through multimodal learning environments: The journey continues. In: C.H. Steel, M.J. Keppell, P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010. pp. 852-863.:853). This point is also supported by Pashler, McDaniel, Rohrer and Bjork; they state "different modes of instruction might be optimal for different people because different modes of presentation exploit the specific perceptual and cognitive strengths of different individuals" (2008:109). This idea is strengthened again by Pashler, et al. (2008), who claim that:

It is undoubtedly the case that a particular student will sometimes benefit from having a particular kind of course content presented in one way versus another. One suspects that educators' attraction to the idea of learning styles partly reflects their (correctly) noticing how often one student may achieve enlightenment from an approach that seems useless to another student (2008:116).

Therefore, it is important to take into account the different students' learning preference in order to select or create suitable activities and materials for the lesson. As an example of the use of modes in the classroom, it is the use of different non-verbal and verbal expressions employed by teachers; the phenomenon is illustrated by Pourhossein, Nizan and Masoumeh (2011POURHOSSEIN, A., NIZAM, H. AND MASOUMEH, S. (2011). The effect of multimodal learning models on language teaching and learning. Theory and Practice in Language Studies, Vol. 1, No. 10, pp. 1321-1327.):

Teachers often use gestures together with speech to draw attention to images and other references within the classroom. In particular, they argue: a variety of modes are interacting and interplaying: gestures, drawings, speech, objects. Each mode contributes to meaning construction: speech to create a difference, an image on the blackboard to get a visual backdrop, manipulation of an object to locate the discussion in the physical setting, action to make clear the dynamic nature of the concept, the image in the textbook to do a stable summary, cohesion is achieved through repetition, synchronization, similarity and contrast. The selection of modes makes meaning: the metaphorical path will be different in each case. Each mode plays a different role in the construction of the entity at hand. Each mode requires the pupils to do a different type of work in order to understand (2011:1325).

1.4.3. Empirical studies on vocabulary acquisition and multimodality

Bisson, Van Heuven, Conklin and Tunney (2014BISSON, M., VAN HEUVEN, W., CONKLIN, K. AND TUNNEY, R. (2014). The role of repeated exposure to multi-modal input in incidental acquisition of foreign language vocabulary. Language Learning. doi:10.1111/lang.12085) found out that students from the University of Nottingham, that had a mean age of 20,7 years old, acquired vocabulary effectively with just two repeated exposures to multimodal stimuli. They conducted an experiment, in which students were exposed to visual, reading, written and aural modes. The students should identify the words after being exposed from two to eight times, according to their groups. The results demonstrated that there was a positive effect on the incidental acquisition of foreign language vocabulary in beginning learners. Royce a college teacher from Columbia University Japan analyzed how TESOL professionals and their students would explore the coexistence of visual, linguistic, and other modes in textbooks, teaching resources and computer screens. Because those modes are used in the classrooms in order to develop multimodal communicative competence, he mentioned some methodological suggestions for analyzing while reading, writing, speaking, listening and vocabulary. In the case of vocabulary he recommended that "the students can immediately associate words with a visual representation, which sets up cognitive associations that facilitate vocabulary learning" (2002:200).

Zarei and Khazaie (2011ZAREI, G. AND KHAZAIE, S. (2011). L2 vocabulary learning through multimodal representations. Procedia - Social and Behavioral Sciences. 15 (1), pp.369- 375.) published a study where they tried to discover how Iranian learners of the English language learn L2 vocabulary through laptop based delivery of multimodal items. In order to carry out the study participants were selected from the ones who were enrolled in EFL classes in an Iranian English language institute. The learners were placed into four different short term memory (STM) ability groups, using the Visual and Verbal Short Term Memory Test. The groups were divided according to their high or low visual and verbal abilities. When finishing the intervention, learners were evaluated on their recognition and recall of vocabulary items. The results indicated that except for low visual and low verbal group, the other three groups treated by vocabulary items with pictorial or written annotations performed significantly better on the tests. The results suggest to consider learners' cognitive styles in teaching vocabulary.

Sydorenko (2010SYDORENKO, T. (2010). Modality of input and vocabulary acquisition. Language learning & technology, 14(2), 50-73.) observed that learners of Russian tend to recognize written word form and the meaning of the word by using captioned videos. However, listening comprehension is facilitated by the use of non-captioned videos. The participants were twenty-six students of the second semester who were learners of Russian. The study delves into the effect of input modality (video, audio, and captions) on (a) the learning of written and aural word forms, (b) overall vocabulary gains, (c) attention to input, and (d) vocabulary learning. The participants were divided into three groups according to different input modality: Group one (N = 8) saw video with audio and captions (VAC); Group two (N = 9) saw video with audio (VA); Group three (N = 9) saw video with captions (VC). At the end of the experiment, all participants completed written and aural vocabulary tests and a final questionnaire. The results indicate that groups with captions (VAC and VC) scored higher on written than on aural recognition of word forms, while the reverse applied to the VA group. The VAC group learned more word meanings than the VA group.

2. METHOD

The current study was conducted as an action research project in order to answer the research question: Does the use of multimodality in the classroom improve the acquisition of vocabulary among 8th, 9th, 10th and 11th graders from two semi-public schools?

2.1. General objective

To determine the influence of multimodality instruction on vocabulary acquisition among 8th, 9th, 10th and 11th graders, from two semi-public schools.

2.2. Specific objectives

  • To determine students' learning preferences in each of the four groups.

  • To identify students' performance on the words learned through the multimodal intervention.

  • To measure the progress on vocabulary acquisition between the pre-test and post-test after multimodal instruction in each of the four groups.

2.3. Participants

One hundred and forty five students from two semi-public schools participated in the present study. At the semi-public school Nº1, there were two groups: eighth and ninth grades, each consisting of thirty one students. In the eighth grade group there were 18 girls and 13 boys, whose ages ranged between 13 and 14. In the ninth grade group, there were 19 girls and 12 boys, and their ages ranged between 14 and 16. In this educational establishment, the students had six hours of English lessons weekly.

In the semi-public school Nº2, there were also two groups: a tenth grade and an eleventh grade that consisted of 38 and 31 students, respectively. In both groups all the participants were boys, and their level of English was roughly pre-intermediate. In the tenth grade group, their age range was from 15 to 17, and they had four hours of English lessons per week. In the eleventh grade group the age range was between 16 and 17, and they had three hours of English lessons weekly.

2.4. Instruments

2.4.1. Vocabulary knowledge scale

The Vocabulary Knowledge Scale (VKS), proposed by Paribakht and Wesche (1997PARIBAKHT, S. AND WESCHE, M. (1997). Vocabulary enhancement activities and reading for meaning in second language vocabulary development. In J. Coady & T. Huckin (Eds.), Second language vocabulary acquisition: A rationale for pedagogy (pp. 174-200). New York: Cambridge University Press.), was adapted to serve as a pre-test and post-test to measure the progress in the learning of the vocabulary presented. It is worth mentioning that, as Wesche and Paribakht (1996, p. 33) state, the purpose of the instrument "is not to estimate general vocabulary knowledge, but rather to track the early development of specific words in an instructional or experimental situation".

The test consisted of thirty words, equally divided in nouns and adjectives that were closely related to the learning units that were taught in each school. The criterion for selecting the words for the test was based on the word bank provided by the Ministry of Education, in the national English language syllabus. In addition, in order to check their frequency, the researchers validated the words in the Collins Online Dictionary (2014) that classified them as: extremely common, very common, in common use and used occasionally. The researchers verified that the words were in the extremely common and very common use classification. Figure 3 details some examples of words used in the Vocabulary Knowledge Scale (VKS) test, which was sat for the students of the four groups.

Figure 3:
Nouns and adjectives for the VKS test.

2.4.2. Visual, Aural, Read/write, and Kinesthetic (Vark) test

The Vark test was developed in 1987 in New Zealand. This evaluation was the first one to systematically present a series of questions that can be easily answered with the aid of a scoring scale, which can be assessed by anyone who can understand the criteria to be used. The acronym VARK stands for Visual, Aural, Read/write, and Kinesthetic sensory modalities that are used for learning information (Fleming and Mills, 1992FLEMING, N. AND MILLS, C., (1992). "Not another inventory, Rather a catalyst for reflection". To Improve the Academy. 11, p.137-155.).

2.5. Action research procedure

To begin with the explanation of the research design, it is important to mention the four stages of the action research approach.

2.5.1. Clarifying vision

It was identified that the most common teaching methods used by teachers to teach English focus mostly on grammar instruction, setting the teaching of vocabulary aside. Consequently, students memorize long lists of words with their corresponding translation in Spanish. These words are not used in context, resulting in students having difficulties to learn new words.

The researchers proposed the acquisition of vocabulary using multimodality exposure in their lessons. The idea behind it is to allow students to perceive and acquire knowledge of words in a more meaningful form. In order to assess students' knowledge of words and identify their learning preferences, the researchers used two instruments: The Vocabulary Knowledge Scale adapted by the researchers (Paribakht and Wesche, 1997PARIBAKHT, S. AND WESCHE, M. (1997). Vocabulary enhancement activities and reading for meaning in second language vocabulary development. In J. Coady & T. Huckin (Eds.), Second language vocabulary acquisition: A rationale for pedagogy (pp. 174-200). New York: Cambridge University Press.) and the VARK (Visual, Aural, Read/write, and Kinesthetic) test (Fleming and Mills, 1992FLEMING, N. AND MILLS, C., (1992). "Not another inventory, Rather a catalyst for reflection". To Improve the Academy. 11, p.137-155.).

2.5.2. Articulating theories

It was decided to plan the actions as follows:

-The lessons were planned based on a task-based approach for language teaching. According to Nunan (2004NUNAN, D. (2004). Task-based language teaching.. Cambridge: Cambridge University Press , p. 4) the task-based approach "involves learners in comprehending, manipulating, producing or interacting in the target language while their attention is focused on mobilizing their grammatical knowledge in order to express meaning and in which the intention is to convey meaning rather than to manipulate form".

-The researchers selected 30 words that belonged to the national English language curriculum for each student group according to the Ministry of Education. In brief, five lessons were planned in addition to a final recycling lesson.

2.5.3. Implementing action and collecting data

The teaching sessions of vocabulary acquisition were implemented using multimodal instruction based on the lesson plans designed. According to Ellis (2003ELLIS, R. (2003). The methodology of task-based teaching. Oxford: Oxford University Press.) there are three common phases for task based: "the first phase is 'pre-task' and concerns the various activities that teachers and students can undertake before they start the task. The second phase, the 'during task' phase, centers on the task itself and affords various instructional options. The final phase is the 'post-task' and involves procedures for following-up on the task performance" (p. 80). The lesson plans the researchers implemented follow the next sequence:

2.5.3.1. Pre-task:

The teacher presented six words through a slide show presentation that contained the words and an image to represent the concept for each word. Then, students had to elicit the meaning of the words based on the images provided, and finally, the teacher showed the word used in context.

2.5.3.2. Task:

In groups, the students had to complete a task that involved the use of the words presented in class. Among the activities that students performed, there were a poster and a comic strip creation, total-physical response activities and the design of a magazine cover.

2.5.3.3. Post-task:

The students worked on a worksheet that contained a vocabulary matching activity. They had to match the presented words with their corresponding definition to consolidate their knowledge.

2.5.3.4. Reflecting on the results

The researchers compared the results of the pre-test and post-test of all the groups in order to analyze the improvement of the students on vocabulary acquisition.

2.6 Type of statistical analysis

In order to analyze the students' vocabulary performance in both semi-public schools, there were two instruments (VKS and VARK) administered at the beginning and at the end of the action research intervention.

The results of both pre-test and post-test were compared to determine if there was improvement in students' vocabulary acquisition after multimodal sessions, based on the Kruskal-Wallis test. To analyze the VARK test, which was related to the students' learning modality, a percentage analysis was used to assess the students' dominant learning preference and their learning preference combinations.

3. RESULTS

3.1 To determine participants learning preferences.

To establish students' learning preferences for each grade group in the two semi-public schools, the VARK test was used. Students answered 16 closed questions; each answer stands for a letter, such as, V for visual, A for aural, R for read or write and K for kinesthetic. The results were evaluated according to the following criteria:

  • Unimodal: a student who has an average of 25% or above in just one of the categories (V, A, R or K).

  • Bimodal: a student who has an average of 25% or above in two categories.

  • Trimodal: a student who has an average of 25% or above in three categories.

  • Multimodal: a student who has an average of 25% or above in two or more than two categories.

The overall results of the four groups were the following

Considering the four groups, 12% of the students were unimodal and 88% multimodal. Within the multimodal criteria, 59% of the students were bimodal, 28% trimodal, and just 1% had the four modes. Based on the results, the action research intervention was designed. According to Fleming and Mills (1992FLEMING, N. AND MILLS, C., (1992). "Not another inventory, Rather a catalyst for reflection". To Improve the Academy. 11, p.137-155., p. 6) "each presentation in another mode gathers in another group of students who might otherwise have missed the point or have been forced to rote learn", referring to the flexibility in using different modes for presenting the contents according to the varied learning preferences in each of the groups.

3.2. To identify students' performance on the words learned through the multimodal intervention

In order to analyze the results of the words presented to students in terms of high scores for correct answers and those words that were almost unknown for them, the researchers considered each word's mean value. To narrow the results, there were selected the three most well-known words and the three least well-known words by each group. The results of the 8th grade group were the following:

In the 8th group, the three words that were well-known by the students in the pre-test increased their mean score, while the least known words decreased their mean values during the posttest administration (See Figure 5). As for the results of the 9th grade, the mean values are:

Figure 4:
All grade groups' learning preferences.

Figure 5:
The most and least well- known words for the 8th grade group.

Based on Figure 6, the least known words increased their mean values considerably, but two of the words selected as the most well-known words slightly decreased their mean scores in the post-test. However, the decrease would be considered minimal because the differences are just 0,15 points. For the 10th grade group, the results are:

Figure 6
The most and least well- known words for the 9th grade group.

In Figure 7, the six words selected increased their mean scores, being the most noticeable growth for the least known words, such as, prompt, which increased 0,95 points. The results of the 11th grade group were the following:

Figure 7:
The most and least well-known words for the 10th grade group.

In Figure 8, it can be observed that the 11th grade group either increased their mean values or maintained them. The word favorite maintained its mean score, and the higher mean value increase was for the word profit.

Figure 8:
The most and least well- known words for the 11th grade group.

3.3. To measure participants' overall performance on vocabulary acquisition in the pre-test and post-test after multimodal instruction.

One of the main goals of this study was to establish the performance of the four groups in the pre-test and post-test after the multimodal intervention. According to the descriptive statistics, the results for the pre-test were the following:

Regarding the Figure 9 data, in the pre-test the 8th grade group had a better performance in comparison to the other three groups, having a mean score of 79,55. Consequently, the 8th grade group obtained the highest minimum (69 points) and maximum scores (113 points) in the VKS test. In spite of the fact that the 11th grade group got the lowest mean score (62, 87 points), it had a maximum score of 104 points in the test, which was higher than the 9th and 10th grade groups. According to the descriptive statistics, the results for the post-test were the following:

Figure 9:
Four groups pre-test descriptive statistics.

With regard to the post test, the 8th grade group again exhibited the highest mean score of 86,58; in addition, it had the highest minimum score but not the maximum one. In contrast, the 9th grade group obtained the lowest mean score of 76,29 among the four groups. It is important to notice that the 11th grade group showed a remarkable progress, upgrading 19,36 points in the mean score; subsequently, it got the maximum score of 137 points in the post-test. Taking into account the overall performance of the four groups, it can be observed in Figure 11 and Figure 12 that the mean score of the four groups increased 15,02 points.

Figure 10:
Four groups post-test descriptive statistics.

Figure 11:
All grade groups pre-test descriptive statistics.

Figure 12:
All grade groups post-test descriptive statistics.

The non parametric Kruskal Wallis test was used to corroborate the progress of the four grade groups as a whole and to check if there was a statistically significant difference. For this purpose, it is needed to observe the tests' results in Figure 11 and Figure 12, and notice if the asymptotic significance is less than 0,05. It is worth mentioning that the decision rule is: If p < 0,05, HO is rejected.

Based on Figure 13 and Figure 14, it is observed that there is an overall improvement between the pre and post-tests of the four grade groups, because the asymptotic significance lies within the range of 0,05. Therefore, the null hypothesis is rejected. In other words, there is a statistically significant difference between the pre-test and post-test after the multimodal intervention.

Figure 13:
Four grade groups pre-test statistics

Figure 14:
Four grade groups post-test statistics

FINAL REMARKS

The use of multimodality in the classroom context significantly benefits the students in their process of learning new vocabulary. This statement is supported by the data obtained, in which after comparing the results of the pre-test and the post-test, it was determined that there was a statistically significant difference. The difference of the mean scores among the four groups between the pre-test and post-test was 15,02 points, which demonstrates that there was an improvement in students' vocabulary acquisition after the multimodal sessions.

The 8th grade group, which obtained the best result in the pre-test (mean: 79,55), got a pre and post test difference of only 7,03 points, which is significantly less than those groups who had a lower results in the pre-test. An example of these groups is the 11th grade group, that obtained 62,87 in the pre-test and achieved a mean score difference of 19,36 points with the posttest. A possible explanation for this is that the 8th grade group initially had a satisfactory level of English compared to the 11th grade group, which had a lower proficiency level in the language.

At an another level of analysis, the results of the Vark test demonstrated that most of the students had multimodal preferences, preferring more than two modes in the test. In addition, the majority of the students presented a kinesthetic learning preference combined with another mode, validating the choice of using the task-based approach as the most suitable for them, because it allows students to be more active in the classroom.

Taking the task-based approach into consideration, it is possible to determine that presenting and involving the students in their learning process highly improved their acquisition of new words. For example, the use of visual aids and productive task activities engaged the students into the content of the lesson and unconsciously helped them to use and remember the words selected for this study. Therefore, the benefits of multimodality constitutes a vehicle of change for teachers' practices.

In the interest of improving English teaching, it seems highly relevant for the researchers to suggest the following aspects: It would be significant to carry out the study in schools from different social backgrounds in order to analyze if multimodality is successful in different contexts. The reason of this recommendation is that the current study was executed in two semi-public schools with similar background characteristics. It may also be relevant for future studies related to this topic to have a control group in order to compare the use of multimodal instruction versus traditional instruction. A reason for this suggestion is that it would be more clarifying in demonstrating the effects of multimodality contrasted with traditional methodologies.

REFERENCES

  • BEZEMER, J. AND JEWITT, C. (2010). Multimodal analysis: Key issues. In: L. Litosseliti(ed), Research methods in Linguistics London: Continuum. pp.180-197.
  • BISSON, M., VAN HEUVEN, W., CONKLIN, K. AND TUNNEY, R. (2014). The role of repeated exposure to multi-modal input in incidental acquisition of foreign language vocabulary. Language Learning. doi:10.1111/lang.12085
  • BLOOM, L. (2000). The intentionality model of word learning: How to learn a word, any word. In: Golinkoff, R., Hirsh-Pasek, K., Bloom,L. (ed), Becoming a word learner : A debate on lexical acquisition. 1st ed USA: Oxford University Press.
  • CARTER, R. (1998). Vocabulary: Applied linguistic perspectives. London, GBR: Routledge.
  • CHEN, G. AND FU, X. (2003). "Effects of multimodal information on learning performance and judgment of learning". Journal of Educational Computing Research, 29(3), 349-362.
  • ELLIS, R. (2003). The methodology of task-based teaching Oxford: Oxford University Press.
  • FADEL, C. (2008). Multimodal learning through media: What the research says. San Jose, CA: Cisco Systems.
  • FLEMING, N. AND MILLS, C., (1992). "Not another inventory, Rather a catalyst for reflection". To Improve the Academy. 11, p.137-155.
  • GOY, A. (2002). Grounding meaning in visual knowledge In: Coventry, K. & Olivier P., Spatial language: Cognitive and computational perspectives Dordrecht: Kluwer Academic Publishers.
  • GRAVES, M., AUGUST, D. AND MANCILLA-MARTINEZ, J. (2012). Teaching vocabulary to English language learners TESOL International Association/Teachers College Press.
  • GUMPERZ, J. (1999), 'On interactional sociolinguistic method', in S. Saranghi and C. Roberts (eds), Talk, work and institutional order. Discourse in medical, mediation and management settings. Berlin, Mouton de Gruyter, pp. 453-471.
  • HASSETT, D. AND CURWOOD, J. (2009). Theories and practices of multimodal education: The instructional dynamics of picture books and primary classrooms. The Reading Teacher, 63: pp.270-282. doi: 10
  • HENRIKSEN, B. (1999). Three dimensions of vocabulary development. Studies in second language acquisition 21: 303-317.
  • HILL, J. AND MILLER, K. (2013). About classroom instruction that works with English language learners. 2nd edition. Denver: McRel.
  • HUNT, A. AND BEGLAR, D. (2005). A framework for developing EFL reading vocabulary. Reading in a Foreign Language. 17 (1), pp.24-44.
  • KRASHEN, S. (1982). Practice in second language acquisition Oxford: Pergamon Press.
  • KOSNIK, C., ROSWELL, J. AND WILLIAMSON, P. (2013). Literacy teacher educators: Preparing teachers for a changing world Rotterdam, NLD: Sense Publishers, eds. Other Books.
  • LEWIS, M. (ED.). (2000). Teaching collocation Hove: Language Teaching Publications.
  • MAYER, R. (2003). Elements of a science of e-learning., Journal of Educational Computing Research 29(3), pp. 297-313.
  • MEARA, P. (1996a). The vocabulary knowledge framework. [ONLINE] Available at: http://www.lognostics.co.uk/vlibrary/meara1996c.pdf Accessed: January 14, 2015.
    » http://www.lognostics.co.uk/vlibrary/meara1996c.pdf
  • MEARA, P. (1996B) The dimensions of lexical competence. In: G Brown, K Malmkjaer and J Williams (eds.) Performance and competence in second language acquisition. Cambridge: Cambridge University Press. 1996. pp.35-52.
  • MEHRING, J. (2005). Developing vocabulary in second language acquisition: From Theories to the Classroom. Available at: http://www.hpu.edu/CHSS/LangLing/TESOL/ProfessionalDevelopment/200680TWPfall06/03Mehring.pdf Accessed: March 5, 2015.
    » http://www.hpu.edu/CHSS/LangLing/TESOL/ProfessionalDevelopment/200680TWPfall06/03Mehring.pdf
  • NAKATA, T. (2006). Implementing optimal spaced learning for English vocabulary learning: Towards improvement of the low-first method derived from the reactivation theory. The JALT CALL Journal, 2(2)
  • NATION, P. (2001). Learning vocabulary in another language. Cambridge: Cambridge University Press
  • NUNAN, D. (2004). Task-based language teaching. Cambridge: Cambridge University Press
  • O'DELL, F. (1997). "Incorporating vocabulary into the Syllabus". In N. Schmitt & M. McCarthy (Eds.), Vocabulary: description, acquisition and pedagogy.. Cambridge: Cambridge University Press
  • PARIBAKHT, S. AND WESCHE, M. (1997). Vocabulary enhancement activities and reading for meaning in second language vocabulary development. In J. Coady & T. Huckin (Eds.), Second language vocabulary acquisition: A rationale for pedagogy (pp. 174-200). New York: Cambridge University Press.
  • PASHLER, H., MCDANIEL, M., ROHRER, D. AND BJORK, R. (2008). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9(3), 105-119.
  • PICCIANO, A. (2009). Blending with purpose: The multimodal model. Journal of the Research Centre for Educational Technology, 5(1), p. 4-14.
  • POURHOSSEIN, A., NIZAM, H. AND MASOUMEH, S. (2011). The effect of multimodal learning models on language teaching and learning. Theory and Practice in Language Studies, Vol. 1, No. 10, pp. 1321-1327.
  • READ, J. (1993). The development of a new measure of L2 vocabulary knowledge. Language Testing, 10, pp.355-371.
  • ROYCE, T. (2002), Multimodality in the TESOL classroom: Exploring visual-verbal synergy. TESOL Quarterly, 36: 191-205. doi: 10.2307/3588330.
  • SANKEY, M., BIRCH, D. AND GARDINER, M. (2010). Engaging students through multimodal learning environments: The journey continues. In: C.H. Steel, M.J. Keppell, P. Gerbic & S. Housego (Eds.), Curriculum, technology & transformation for an unknown future. Proceedings ascilite Sydney 2010. pp. 852-863.
  • SYDORENKO, T. (2010). Modality of input and vocabulary acquisition. Language learning & technology, 14(2), 50-73.
  • THORNBURY, S. (2002). How to teach vocabulary Harlow: Longman.
  • UNSWORTH, L. (2008). Multimodal semiotics: Functional analysis in contexts of Education. London, GBR: Continuum International Publishing.
  • WESCHE, M. AND PARIBAKHT, S. (1996). Assessing vocabulary knowledge: depth versus bread. Canadian Modern Language Review, 53,1.
  • WILKINS, D. (1972). Linguistics in language teaching London: Arnold. Word [Def. 1]. (n.d.). Oxford Dictionary Online. In Oxford Dictionary. Available at: http://www.oxforddictionaries.com/us/definition/americanenglish/word?SearchDict Code=all Assessed: December, 15, 2014.
    » http://www.oxforddictionaries.com/us/definition/americanenglish/word?SearchDict Code=all
  • WYSOCKI, A. (2002). Teaching writing with computers: An introduction (3rd ed.). Boston: Houghton-Mifflin. p. 182-201.
  • YELLAND, N., LEE, L. AND O'ROURKE, M. (2008). Rethinking learning in early childhood education. Berkshire, GBR: McGraw-Hill Education.
  • YONGQI GU, P. (2003). Vocabulary learning in second language: Person, task, context and strategies. TESL-EJ, 7(2).
  • ZAREI, G. AND KHAZAIE, S. (2011). L2 vocabulary learning through multimodal representations. Procedia - Social and Behavioral Sciences 15 (1), pp.369- 375.

Publication Dates

  • Publication in this collection
    Jan-Apr 2016

History

  • Received
    04 Sept 2015
  • Accepted
    21 Mar 2016
UNICAMP. Programa de Pós-Graduação em Linguística Aplicada do Instituto de Estudos da Linguagem (IEL) Rua Sérgio Buarque de Holanda, n.571 - Cidade Universitária - CEP: 13083-859, Telefone: (+55) 19 - 3521-6729 - Campinas - SP - Brazil
E-mail: tla@unicamp.br