Assessing undergraduate students' reading process: insights from questionnaire and reading task data

Avaliação do processo de leitura de alunos universitários com base em dados de questionários e de tarefas de leitura

ARTIGO

Assessing undergraduate students' reading process: insights from questionnaire and reading task data¹

Avaliação do processo de leitura de alunos universitários com base em dados de questionários e de tarefas de leitura

Claudia Finger-Kratochvil*; Rosane Silveira**

*Universidade do Oeste de Santa Catarina. E-mail: [e-mail];

**Universidade Federal de Santa Catarina. E-mail: [e-mail]

ABSTRACT

Many institutions have been studying the construction of different aspects of the reading process and the reader (e.g. OECD, INEP), and they have revealed a gap in the process of building reading abilities at all levels of education. The present study focuses on entry-level college students and analyzes data from thirty-three students, collected by means of (a) two questionnaires assessing the participants' views of the reading process, purposes of reading, and their reading practices, and (b) three reading units designed to measure the participants' reading ability in their native language. The results revealed that a large number of students spend little time reading, although they report that reading is a rewarding activity. Moreover, for most of them, reading is a bottom-up process, and the consequences of this view can be observed in their performance on the reading tasks.

KEYWORDS: reading; assessment; college students.

RESUMO

Várias instituições têm-se preocupado com a qualidade da formação do leitor (e.g. OECD, INAF) e têm apontado a construção lacunar do leitor. O presente estudo investiga alunos universitários, utilizando dois questionários que permitiram examinar as visões que os participantes têm do processo de leitura, os objetivos da leitura, bem como suas práticas de leitura. Também foram utilizadas três unidades de leitura desenvolvidas para avaliar a habilidade de leitura dos participantes na língua materna. Os resultados mostram que muitos alunos dedicam pouco tempo à leitura, apesar de reconhecer que essa atividade resulta em ganhos pessoais. Ressalta-se ainda que a maioria dos participantes concebe a leitura como um processo ascendente (botton-up), o que se confirma no desempenho que demonstram nas tarefas de leitura.

PALAVRAS-CHAVE: leitura; avaliação; alunos universitários.

Introduction and Review of the Literature

Reading is a complex task that is influenced by a number of factors (i.e., lexical knowledge, familiarity, text genre, background knowledge activation, attention) related to the text, the reader, and the reading instantiation time (GERNSBACHER, 1997; ZWAAN; SINGER, 2003). To interpret a text successfully, a reader needs to go beyond decoding and establish relations among the different dimensions present in the text and among these dimensions and her/his previous knowledge. Also, the reader needs to consider the context of reading time (the stance) (ROSEMBLATT, 2004). This intricate process enables the reader to construe meaning and is the basic requirement for comprehension and for knowledge construction and retention through reading. Moreover, comprehension only takes place if there is a strong connection between the reader and the text, and if certain conditions are satisfied during the reading process. Some of these are, for instance, schemata activation, which consists of generalized mental representations of complex patterns of behaviors, places, texts and events built by each person, representing her/his organized and continuously updated background knowledge (ANDERSON, 2004; CARRELL, 1992).

As our experience and knowledge as readers grow, we most likely analyze texts by relying on categories previously elaborated throughout the distinct transactional reading situations in which we have been engaged (ROSEMBLATT, 2004). For example, when someone is reading s/he identifies distinct elements that will approximate or separate the text, the reader, and a particular reading context. This process allows for making abstractions and (re)elaborating knowledge in order to compose mental representations that will be part of our memory. These processes will always be part of reading activities. The relationship between the text (and the information it carries), the reader (and all her/his knowledge and experience) and the reading context (which changes constantly) plays an important role and determines how successful (re)construction of meaning and comprehension processes will be.

Reading assessment initiatives conducted by the Brazilian government (e.g. Prova Brasil) and by non-governmental organizations (Instituto Paulo Montenegro, 2001, 2003, 2005, 2007) have demonstrated a serious gap in the reading and writing abilities of primary school students² (ages 7-14) and 15-64 year-old Brazilians. As reported by Bencini and Minami (2006), the scores obtained by students concluding primary school (8^th grade) were at the level expected for students concluding elementary school (4^th grade), thus demonstrating that the efforts to develop literacy in primary school have been far from successful if we consider the current socioeconomic and cultural demands and the goals established by OECD - Organization for Economic Co-operation and Development (2002a, 2002b). This finding reveals a major threat to these students' professional education and personal growth because being able to read well is a sine qua non condition for development at any educational level and because reading is an ability required throughout life. Furthermore, a crucial question is whether students entering college possess the necessary levels of literacy to cope with the demands of both higher education and lifelong learning.

The present study draws from the literature in the literacy field (e.g., AGUIAR, 1983; STREET, 1984; OECD, 2002a, 2002b; SOARES, 1992; 2003; RIBEIRO, 2003), as well as studies regarding the cognitive and metacognitive processes in which a reader engages (BAKER; BROWN, 1984; AFFLERBACH, 1990; CARRELL, 1992; GARNER, 1992; TRABASSO et al., 1995; PERFETTI, 1997; ANDERSON, 2004). This paper also extends previous research investigating the reading ability of entry-level college students conducted by Finger-Kratochvil, Klein, and Baretta (2004, 2005). The literature has highlighted the multifaceted and complex nature of literacy development, in addition to stressing the importance of explicit and systematic instruction on the skills involved in learning to read and to write. The main purpose here is to investigate the reading practices and views of entry-level college students in Brazil and their reading-comprehension performance on informative texts. This study will contribute information on reading in academic scenarios and on the extent to which the reading literacy level of students who finish high school matches the standards proposed by the Programme for International Student Assessment (PISA) sponsored by the OECD (2002a, 2002b).

According to Finger-Kratochvil, Baretta, and Klein (2005), who investigated the reading experience of entry-level college students from a small town located in the south of Brazil, most of these students possess basic reading abilities and skills. This means, for example, that they are capable of working with information that is explicitly presented and of recognizing the author's purpose in texts about familiar topics when the required information is prominent in the text. However, when confronted with more complex tasks that demand working with competing information and inferences, that is, when cognitive processing at more complex levels is required, low performance rates are obtained, thus revealing the difficulties faced by the students.

This situation is alarming because reading assignments at college level are far more complex, numerous, and demanding than they are in high school. The required course reading contains polysemous words and jargon, not to mention that students are expected to master a number of concepts (FRANCIS; SIMPSON, 2003). Undoubtedly, reading imposes challenging obstacles for entry-level students who are lagging behind in terms of reading ability. The present study of a sample of southern Brazilian students entering a social sciences program investigates whether these students understand the importance of developing their reading ability, and whether their performance on three reading units indicates that they are prepared to deal with the reading assignments they will face as college students.

Methodology

This section describes the participants in this study, the research instruments (questionnaires and reading units), and the procedures for data collection and data analysis.

The 33 participants in this study come from an original pool of 53 students recently admitted to a Social Sciences undergraduate program at a university located in Santa Catarina state, in the south of Brazil. The participants were invited to contribute to this study and all of them signed a letter of consent in the first data collection meeting. Further information about the participants will be provided in the results section, as one of our research questions involves examining their profile.

The study relied on three rounds of data collection sessions. The materials used to gather the data were: (a) a questionnaire addressing socio-economic and reading issues, (b) a questionnaire investigating the participants' reading practices and views, and (c) three reading units designed according to the guidelines proposed by PISA (OECD, 2002, 2003).

The first questionnaire contained 26 questions, most of them including sub-items and designed in a closed format that provided the participants with a range of possible answers. Two exceptions were questions 25 and 26, which were designed in an open format to inquire about the participants' perception of reading and their definition of a good reader. Most of the remaining questions were aimed at gathering biographic and socio-economic information about the participants, as well as their reading views and practices.

The other questionnaire inquired about the participants' reading practices, views, and strategies. This document contained 36 closed questions. Some of the questions were very similar to the ones in the first questionnaire, but were worded differently. This repetition was intentional as it allowed the researchers to compare information from both questionnaires in order to verify the consistency of the responses.

To assess the participants' reading ability, the researchers employed three reading units (hereafter RU) which were designed according to the guidelines proposed by PISA (OECD, 2002b), which characterize reading ability in three domains: Retrieving Information, Interpreting Information, and Reflection and Evaluation. Each domain encompasses five levels of complexity, ranging from least difficult (1) to most difficult (5). The questions for each of the three RUs were designed either in a multiple choice format or as open items.

TAB. 1 provides information about the different domains and levels of difficulty for each question in the three RUs designed for the present study. The number of questions included in each RU varied: RU01 had six questions (maximum score: 2,862), RU02 had four (maximum score: 2,065), and RU03 had eight (maximum score: 4,962). The questions were classified according to their level of complexity for each domain. Note that some questions are repeated in the table. This is the case for questions that could receive partial credit (score 1) or full credit (score 2). Thus the maximum score for the three RUs combined is 9,889. The calculation of the final scores for each RU was established according to the possible highest score to be achieved within the RU. Thus, for questions that had two score levels, we considered the highest score (2) only. For example, considering all questions from RU01, the maximum total score was 2,862. Imagining that a participant successfully completed all the tasks for RU01 but received only partial credit for Q06 (score 1=380), instead of receiving full credit (score 2=420), this participant would obtain a final score of 2,822, which is 40 points less than the maximum score for RU01, as can be observed in the calculation below:

The scores were also calculated taking into account the level of complexity of the questions, which were used as a measure of the participants' reading ability. In other words, we considered the score a participant could obtain for a specific level and if this participant answered correctly all the questions within that level, even if some of them received score (1). Thus, a maximum score was devised for each level, and the participants could obtain different scores within each level, as shown in TAB. 2. The last column in this table shows the cumulative scores, meaning that the scores obtained for the questions of each level were added up to yield the total score of all RUs combined. For instance, if a participant answered correctly all the questions of all RUs within level 1 (RU01Q06 (score 1), RU02Q01, RU01Q01, RU03Q05), s/he would receive the maximum score for this level, which is 1,594. If the same students also answered correctly all the questions within level 2 (RU01Q6 (score 2), RU01Q04 (score 1), RU01Q05), s/he would obtain a score of 2,574, i.e., 1.594 (level 1) plus 980 (level 2).

Each RU contained an informative text which had been published in Brazilian newspapers or magazines with national circulation, and also a few reading tasks. The text in RU01 was published in a mainstream weekly magazine (Veja, May 10^th, 2006) and discussed a topic related to Brazilian history. This text was accompanied by six tasks distributed across the three domains and levels of difficulty 1-3 (see TAB. 1).

The second RU included a text about new technological trends and was published in a major Brazilian newspaper (O Estado de São Paulo, August 15^th, 2005). The text for RU02 was accompanied by four tasks that belonged to the Interpreting Information and Reflection and Evaluation domains. As shown in TAB. 1, the levels of difficulty of these questions ranged from levels 1-2 and 4-5.

Finally, RU03 was based on a text in the area of botany, published in a Brazilian popular science magazine (Ciência Hoje, June, 2003, vol. 33, n.194). There were eight tasks in this unit, including at least one question for each domain, and a level of difficulty ranging from 4 to 5 (see TAB. 1).

The three texts selected for the reading units were subject to an intelligibility analysis to ensure the participants would be comfortable with the chosen texts. Previous studies have shown that this type of analysis is both necessary and difficult to accomplish, since defining relevant and consistent criteria to rate the levels of different texts is a complex endeavor (HARRISON, 1980; ZAKALUK; SAMUELS, 1988; STAHL, 2003). For this reason, these studies have assessed the intelligibility of texts by examining their density, as proposed by Eggins (2004), thus yielding levels of lexical and syntactic density.

Lexical density is calculated in terms of percentage of words in a text that are content words (nouns, verbs, adjectives, and adverbs), while syntactic density (also called grammatical intricacy) involves counting the number of sentences and clauses, and then dividing the number of clauses by the number of sentences in a text (EGGINS, 2004; HALLIDAY; MATTHIESSEN, 2004).

According to Eggins (2004), the higher the lexical density and the lower the syntactic density of a text, the more difficult it is for the reader to process. Eggins explains that written texts are characterized by having high lexical density and low syntactic density, while spoken texts possess low lexical density and high syntactic density. A written text that has many clauses per sentence is said to have high syntactic density, meaning that its syntax is similar to that of a spoken text, and that this type of written text is expected to be easier to process.

The three texts used in this study possess a similar level of lexical density (RU01: 51,45; RU02: 51,04; RU03: 56,23), corresponding to the nominalization of written texts scores, however text RU03 had a slightly higher number of content words, which might indicate that it is slightly more difficult to process than the other texts. Concerning syntactic density, the RU01 text (2,15) had the lowest level, followed by the RU02 text (2,68) and the RU03 text (3.10). This means that the RU01 text should be more difficult to process than the other texts, although its lexical density is similar to the RU02 text. In conclusion, we can say that the RU02 text is expected to be the least difficult, taking into account only the text, not the tasks accompanying it, while the RU03 text is expected to be the most demanding. However, if we consider both texts and tasks, the degree of difficulty for each of them is different: RU01 is the easiest, while RU03 is the most difficult (see TAB. 1 to check levels and domains).

The procedure for data collection included the following steps:

(a) signing the letter of consent, completing Questionnaire I, completing RUO1; (b) completing RU02; (c) completing Questionnaire II; and (d) completing RU03. The number of participants fluctuated throughout the four sessions (from 53 to 33), as the school semester was ending and the participants were taking their final exams by the time sessions 2 and 3 took place. Thus, only 33 participants completed all questionnaires and RUs, and in the Results and Discussion section, only the data from these participants will be taken into consideration.

The data obtained through the questionnaires were tabulated and analyzed in order to examine the typical profile of undergraduate students who entered the social sciences program. This information was also analyzed to examine their reading views and practices. The RUs provided information about the participants' reading performance, and these data were analyzed to check if this performance was influenced by the reading domain and complexity level of the comprehension questions.

Results and Discussion

In this section we report and discuss the results obtained in the present study. The section was organized around four research questions that guided the study. First we present the participants' profile. The next step is to discuss how the participants view the reading process and describe their reading practices. Then, we report on the results of the participants' performance on the three RUs separately. Finally, we analyze the degree of difficulty posed by the questions included in the three reading units, discussing the extent to which the reading domain and complexity level of the questions influenced the participants' responses.

Typical profile of undergraduate students who enter the social sciences program

The data obtained through the biographical questionnaire were analyzed in order to provide a better understanding of the participants' profiles. The results indicate that the participants are mostly young adults, with ages ranging from 18 to 30. The mean (20.28, sd = 3.03) indicates that most of them entered the university within 3 years of leaving high school, since in Brazil the typical age of students concluding high school is 17 or 18.

Regarding gender, 64% of the participants are female, while 36% are male. This fact corroborates a trend in the Brazilian education system already attested by the UNICEF Report (2003), i.e., more women attend school. This is in contrast with the world trend in which female students have less access to education (UNICEF, 2003).

Another important characteristic of our participants is the fact that most of them report having either a part-time or a full-time job (72.2% and 15.1%, respectively). Thus, most participants need to cope with a routine that involves working during the day and attending college in the evening. This seems to be the reality of many students attending private or foundational colleges and universities in Brazil, which generally offer courses in the evening in order to cater for students who work. Unfortunately, there are no official statistics delineating the profile of these students in Brazil.

The questionnaires also provided information concerning the schooling level of the participants' parents, revealing that most of them have finished elementary school (mothers: 100%, fathers: 94.3%), but only about half of them have concluded primary school (mothers: 58.9%, fathers: 50.9%). This scenario suggests that many participants live in a household where the levels of literacy are low and reading practices are most likely limited.

Participants' reading views and practices

In addition to biographical information, the questionnaires provided information about how the participants view reading, as well as their reading practices. TAB. 3 shows how the participants responded to a set of questions assessing their reading views.

Nearly 70% state they read spontaneously, not only when required. Though this response suggests a positive view of reading, the next one shows that 65.18% of them reported that reading is not amongst their favorite pastimes; while 60.42% stated they dislike discussing books with other people. About 40% of the participants reported finding it difficult to finish reading a book, and nearly 55% of the participants stated they like receiving books as presents. When asked whether reading is a waste of time, most participants disagreed (88.68%). However, only about 50% reported that they like going to libraries and bookstores, and about the same percentage only read to find useful information. In addition, about 40% of the participants reported finding it difficult to sit down and read for more than a few minutes.

As explained in section 2, different questions were used to scrutinize the participants' reading views. An open question (Q25) was provided to investigate what reading means for the participants. Most of them tended to provide short answers to this question, and the researchers relied on these answers to devise categories that will be discussed in the following paragraphs.

As TAB. 4 shows, eight categories were devised for the participants' responses about what reading means. Some of these are connected to events that the participants notice in the reading process, while others are related to reading purposes. The categories belonging to the first group obtained low frequency rates: mental activity, practice, decoding, comprehension, and extracting information or interpreting. On the other hand, the categories related to reading purposes tended to be mentioned more frequently by the participants. Thus, 42.2% of the participants reported that reading means learning, while 10.8% declared that reading should be for pleasure or entertainment.

Taken together, the participants' responses to Q25 (an open question) are in agreement with the overall results obtained with the multiple choice questions used to scrutinize the participants' reading views. The results indicate that the participants seem to believe that reading is an important way of improving their knowledge, but many of them see reading as an obligation imposed by school, not as a source of pleasure and entertainment.

A further analysis involved examining the participants' answers to another open question (Q26), which asked them to define a good reader. Based on their answers, nine categories were devised, as can be seen in TAB. 4. Many participants (21.5%) reported that a good reader is someone who reads for pleasure. Other frequent answers were that a good reader aims at comprehending texts (15%), reads frequently (15%) and is capable of selecting the material that should be read (12.5%). Other features of a good reader were pointed out by the participants, albeit less frequently: concentration, variation of reading genres, ability to go beyond the decoding process (5.1% each) and ability to extract the necessary information from the text (4.4%). A few participants believe that good readers are people who share their ideas and interpretations about their reading with other readers (2.9%) or who read a lot of texts (1.5%)

In addition to looking into the participants' views on reading, the questionnaires were also designed to reveal their reading practices. Regarding this issue, the participants were asked (a) whether they spend time reading for pleasure (daily), (b) how often they read different genres, (c) how much access they have to books at home, and (d) how often they borrow books from libraries.

Nearly half of the participants reported reading for pleasure for 30 minutes or less per day, while 26.4% stated not reading for pleasure at all. Of the 33 participants, only 13.2% declared reading for 30 minutes to 1 hour for pleasure, while only 7.5% read for this purpose from 1 to 2 hours and no participant reads for pleasure for 2 hours or more. These results are not surprising, as the participants' responses to Q25 (previously discussed) revealed that reading is not a source of entertainment for nearly 70% of them, although nearly 22% of them defined a good reader as someone who reads for pleasure and entertainment.

Another reading practice issue addressed by the questionnaires was how often the participants read different text genres. TAB. 5 shows that electronic messages and texts available on the internet are the most popular genre, followed by newspapers and magazines. In view of the previous results concerning the participants' reading practices and stances, it was no surprise that very few participants reported reading genres that are more related to entertainment, such as comic books and fiction.

Another important fact regarding reading practices is how much access the participants have to books at home. The low literacy level of their parents, as well as the limited access to books at home, reflect the socioeconomic educational background of the household of the participants (OECD, 2002b). As TAB. 6 shows, a notable 37.7% of the participants stated they have only 1-10 books at home, and 41.50% declared having 11-50 books.

Moreover, as the data in TAB. 6 shows, when asked how often they borrowed books from libraries, the participants' responses revealed that as many as 22.6% stated that they never or almost never check out books, while 43.4% do a few times a year.

The figures in TAB.s 5 and 6 are important as they reveal the low literacy level of the participants' household, and several studies have indicated that family literacy plays a major role in promoting the value and cultivation of reading abilities, i.e., influencing students' attitudes to reading, education, and their future goals (OECD, 2002b; BENCINI; MINAMI, 2006). These results help us understand better why for many participants reading is not an important source of entertainment, as limited access to books at their homes may lead them to believe that reading is an activity mainly connected to school. Nonetheless, motivating students to engage in reading activities for pleasure and helping them develop their reading abilities is a responsibility that should be shared by the family and the school. These results suggest that neither family nor school has succeeded in nurturing a positive view of reading.

If we analyze the types of texts and tasks present in many textbooks used in Brazil, we can see that they still include short texts or fragments of large texts (FINGER-KRATOCHVIL, 1997). Moreover, the reading activities accompanying these texts are focused on retrieving information, the reading domain in which students perform better (FINGER-KRATOCHVIL; BARETTA; KLEIN, 2005). Another major problem is that learners seem to have little motivation to read longer texts outside the school context, and the family environment fails to promote reading development. Sadly, participants report a bottom-up conception of reading in which reading successfully means being able to extract information from the text. Possible implications of these findings are that there is an overall limited view of reading, both at home and at school. Thus, reading is not regarded as a source of entertainment, but merely as a source of information, and the reader's task consists of simply retrieving this information from the text.

Participants' performance on the reading units

This section presents the results of the participants' performance on the three RUs. First, we present the results for each participant on each RU, and on the three RUs combined. Next, we turn our attention to the analysis of the results in terms of reading domains and difficulty levels of the questions accompanying each RU.

In order to assess the participants' reading ability, three RUs were designed. RU01 was expected to be the easiest, followed by RU02, while RU03 included more complex text and tasks (see details in Section 2). TAB. 7 summarizes the results for the three RUs individually and combined, presenting the frequencies and the descriptive statistics.

As can be seen, in RU01 a number of participants obtained the same score, with more than 50% scoring between 2,337 and 2,862. This indicates that the tasks were relatively easy for many participants. Note that five participants obtained the maximum score (2,862). One exception was participant CA0614, who scored very low (535), meaning that s/he got less than 20% of the questions right.

A similar result was obtained for RU02, in which more than 50% of the participants scored between 1,480 and 2,065. Note that another five participants received the maximum score (2,065), and that two of them also got the maximum scores in RU01 (CA0649 and CA0659). Conversely, three participants got extremely low scores (405-525), indicating that they responded correctly to less than 20% of the questions. Note that one of these participants with low scores obtained the maximum score for RU01 (CA0658), while the participant with the lowest score in RU01 (CA0614) is among those with the lowest scores in RU02 as well.

On the other hand, a wider range of scores was found for RU03, as well as for the total score of the three RUs combined. In RU03 no participant obtained the maximum score (4,292), which can be attributed to the fact that this unit was more complex in terms of text and tasks than the other two units. Nevertheless, two participants (CA0655 and CA0671) got close to the maximum score, thus answering correctly about 85% of the questions. One participant (CA0616) obtained a very low score (685), which indicates that only about 14% of his/her answers were correct. The same participant performed poorly on RU01, but obtained the 6^th highest score on RU02.

Analyzing the results for the three RUs combined, we can see that the group's mean score is 6,261.88 (SD = 1,277.36), and this corresponds to 63.32% of correct responses. Overall the results for the RUs indicate that the participants' reading ability varies considerably, with nearly 15% of the participants obtaining between 7,989-8,644 score points (out of a possible total of 9,889), which is between 80-87% of correct responses. However, for about 15% of the participants the RUs proved difficult, since they obtained low scores ranging from 3,527 to 4,954, which represents about 35-50% of correct responses.

Based on the standards established by PISA (OECD, 2002, 2003), the results of the RUs combined indicate that the participants' reading ability is within level 4, since their mean score equals 6,261.88 and this value is within the range of level 4 (from 4,749 to 7,804). However, it is important to point out that the participants' mean score is a long way from the maximum score for level 4, which is 7,804. A total of twenty three participants (69.7%) belong to level 4, five (15.15%) belong to level 3, and five to level 5 (15.15%). Thus, we can conclude that the reading proficiency level of most of these participants is below the objective established by PISA guidelines for students who have accomplished eleven years of schooling—level 5—since nearly 85% of them are below this level.

The data displayed in TAB. 8 were also analyzed in terms of how consistent the participants' performance was across the three tests. Table 8 displays the Pearson correlations for the RUs. These results show a weak, nonsignificant correlation between RUs 1, 2, and 3. However, moderate to strong significant correlations were found among the RUs combined and all the individual RUs. All the correlations were positive, thus indicating that although the participants' performance varied across RUs, overall their reading ability across the three tests tends to be similar. In other words, there is a tendency for the same participants to obtain the highest scores in RU01, RU02 or RU03, or all combined.

Considerations about domains and complexity levels

It is still necessary to discuss how the participants' reading performance interacts with the domains and complexity levels of the questions included in each RU. We begin by analyzing the results for RU01. Note that Q06 belongs to levels 1 and 2 of the Retrieving Information domain, while Q05 belongs to level 2 of the Reflection and Evaluation domain. Therefore, Q06 was expected to be easier than Q05, as the act of retrieving information poses less difficulty than reflecting and evaluating (FINGER-KRATOCHVIL; BARETTA; KLEIN, 2005). Indeed, this expectation was met, as the results in FIG. 1 show that 94% of the participants answered Q6 correctly, albeit 18.18% of these received partial credit for their answers. On the other hand, Q05 caused more problems to participants, and only 24.24% of them managed to answer it correctly. If we take a look at the other questions comprising RU01, we can see that Q1 and Q2, which belong to the Interpreting Information domain, caused little difficulty to the participants; yet Q4 was slightly more difficult and only about 58% of the participants received full credit for their answers, while another 27.27% obtained partial credit. Also, Q03, which belongs to the Reflection and Evaluation domain, was also a highly demanding question for many participants, and 42.42% of them provided wrong answers.

FIG. 2 displays the results for RU02. Here, similar to what happened in RU01, we can see a tendency for one of the questions from the Interpreting Information domain (Q01) to pose fewer problems to the participants, leading to 87.87% correct responses. This was also true for Q04 (66.66% correct responses and 15.15% partially correct responses). Note that Q01 belongs to the Interpreting Information domain, whereas Q4 is part of the Retrieving Information domain. Moreover, Q02, which is in the Reflection and Evaluation domain, proved difficult for 36.36% of the participants, whereas Q03, which is part of the Interpreting Information domain, caused the most problems to participants, thus yielding nearly a 64% rate of incorrect responses. Note that the complexity level of Q03 is 4, which means it is a highly complex question; in fact, this is the question with the highest complexity level in RU2.

FIG. 3 displays the results for the questions in RU03, which is expected to be more difficult than the other RUs since most tasks in RU03 have complexity levels 4 and 5. The results in FIG. 3 show that Q02 yielded the highest rate of correct responses (93.93%). This result can be attributed to the fact that this question belongs to the Interpreting Information domain, which was relatively easy for the participants in the other RUs. However, it should be pointed out that this question has a high complexity level (5), but nevertheless participants managed to answer it correctly most of the time. The remaining questions that belong to the Interpreting Information domain (Q1, Q6, and Q8), all with high complexity levels (4-5), also posed considerable difficulty to the participants, yielding 36.36% incorrect responses, and 45.45% of the participants received only partial credit for Q6.

FIG. 3

The analysis of the participants' performance in view of the domains and complexity levels of the questions corroborates the literature (OECD 2002a, b), which maintains that Reflection and Evaluation domain questions tend to be challenging to less experienced readers. The results showed that the entry-level college students participating in this study obtained the lowest scores when responding to these questions, while the Information Retrieval questions yielded the highest rates of correct responses in most cases. Apparently, the difficulty imposed by the Reflection and Evaluation questions stems from the fact that most participants tended to resort to direct citation of extracts from the texts when completing the reading tasks. This strategy works well with Information Retrieval questions, but not with Reflection and Evaluation questions. These findings corroborate those of previous studies (FINGER-KRATOCHVIL; BARETTA; KLEIN, 2005; PRESSANTO, 2007) which reported that entry-level college students can easily identify and retrieve information that is clearly present in the text, though they tend to struggle with questions that require reflecting, evaluating, and interpreting information.

In addition to the reading ability domains, another factor influencing the degree of difficulty of the questions is their level of complexity. The analysis of the three RUs revealed that the participants tended to perform better on the less complex tasks (levels 1 and 2) than on the more complex tasks (levels 4 and 5). Moreover, the effect of complexity level may overrule the effect of reading domain, given that highly complex questions within the less difficult domains (e.g., Q7 in RU03) yielded low rates of correct responses; similarly, questions with a low complexity level (e.g., Q5 in RU03) but from the most difficult domain—Reflection and Evaluation—led to high rates of correct responses.

Therefore, the results indicate that most participants were capable of dealing with questions that require searching for information that is prominent in the text. However, when the response to a question requires searching for competing information and making more complex inferences, difficulties surfaced, thus revealing the fragility of the participants' reading ability at the time of starting their postsecondary studies. These results for reading performance are in line with the participants' view of reading as an activity closely connected to retrieving information from the text.

Concluding Remarks

The results presented in this paper address only a few of the many factors influencing the construction of the participants' reading literacy. Albeit limited, these results highlight the necessity of further research initiatives in order to obtain a clearer picture of the development of reading literacy at all education levels. We also acknowledge the methodological limitations of collecting data based on questionnaires and the necessity of improving these instruments for future studies.

The reading views and practices reported by our participants, together with their performance on the three RUs, reflect the precarious situation of reading literacy in elementary and secondary education. Working during the day and attending college in the evening is very challenging, especially when students have not reached the necessary level of reading ability required to succeed as college students, as in the case of the participants of this study.

Undoubtedly, this research has barely scratched the surface of the reading problems faced by entry-level college students, yet the results corroborate previous research regarding the gaps in the development of reading literacy in the Brazilian education system (FINGER-KRATOCHVIL; BARETTA; KLEIN, 2005, 2007; PRESSANTO, 2007). Overall, the results reported here indicate that primary and secondary education in Brazil has been partially successful in the development of students' reading literacy, preparing them to decode, identify, retrieve information, and make simple connections, but not to reflect, evaluate, and make complex inferences about the texts they read. As a consequence, these students might face major challenges while completing the reading assignments they are given in college and even their professional success in their current or future jobs may be jeopardized.

References

AFLLERBACH, P. P. The influence of prior kn owledge on expert readers' main idea construction strategies. Reading Research Quarterly, v. 25, n. 1, p. 31-46, 1990.

AGUIAR, V. T. A leitura nos currículos oficiais. Leitura: Teoria e Prática, v. 2, n. 2, p. 31-33, 1983.

ANDERSON, R. C. Role of the reader's schema in comprehension, learning and memory. In: RUDDELL, R. B.; UNRAU, N. J. (Ed.). Theoretical Models and processes of reading. Newark: IRA, 2004. p. 594-606.

BAKER, L.; BROWN, A. L. Metacognitive skills and reading. In: PEARSON, P. D. (Ed.). Handbook of research on reading. New York: Longman, 1984. p. 353-394.

BENCINI, R.; MINAMI, T. O desafio da qualidade. Nova Escola, v. 196, n. 21, p. 40-45, 2006. CARRELL, P. L. Awareness of text structure: effects on recall. Language Learning, v. 42, n. 1, p. 1-20, 1992.

EGGINS, S. An introduction to systemic functional linguistics. London/New York: Continuum International Publishing Group, 2004.

FINGER-KRATOCHVIL, C. Cartilhas: auxílio ou empecilho para o letramento? 1999. 254 f. Dissertação (Mestrado em Linguística) - Departamento de Linguística. Universidade Federal de Santa Catarina, Florianópolis, 1997.

FINGER-KRATOCHVIL, C.; KLEIN, L.; BARETTA, L. As habilidades de leitura dos alunos calouros da Unoesc-Joaçaba: um estudo diagnóstico. CONGRESSO DE LÍNGUA E LITERATURA, 4, 2004, Joaçaba. Anais... Joaçaba: Ed. UNOESC, 2004.

FINGER-KRATOCHVIL, C.; BARETTA, L.; KLEIN, L. Habilidades gerais de leitura de alunos calouros: um estudo diagnóstico. CONGRESSO DE LEITURA DO BRASIL, 15, 2005, Campinas. Anais... Campinas: ALB/ UNICAMP, 2005.

FINGER-KRATOCHVIL, C. Aluno-calouro-leitor e as exigências da formação universitária: como se apresenta essa relação? CONGRESSO DE LEITURA DO BRASIL, 16, 2007, Campinas. Anais... Campinas, SP: ALB/UNICAMP, 2007.

FRANCIS, M. A.; SIMPSON, M. L. Using theory, our intuitions, and a research study to enhance students' vocabulary knowledge. Journal of Adolescent & Adult Literacy, v. 47, n. 1, p. 66-78, 2003.

GARNER, R. Metacognition and self-monitoring strategies. In: SAMUELS, S. J.; FARSTRUP, A. E. (Ed.). What research has to say about reading instruction. Newark, DE: International Reading Association, 1992. p. 236-252.

GERNSBACHER, A. M. Two decades of structure building. Discourse Processes, v. 23, p. 265-304, 1997.

HALLIDAY, M. A. K.; MATTHIESSEN, C. An introduction to functional grammar. London: Hodder Arnold, 2004. HARRISON, C. Readability in the classroom. Cambridge: Cambridge University Press, 1980.

INSTITUTO PAULO MONTENEGRO. Inaf: 1^o indicador nacional de alfabetismo funcional: um diagnóstico para a inclusão social pela educação. São Paulo: Instituto Paulo Montenegro, Ibope Opinião e ONG Ação Educativa. Disponível em: <http://www.ipm.org.br/an_ind_inaf_1.php>. Acesso em: out. 2008.

INSTITUTO PAULO MONTENEGRO. Inaf: 3^o indicador nacional de alfabetismo funcional. São Paulo: Instituto Paulo Montenegro, Ibope Opinião e ONG Ação Educativa. Disponível em: <http://www.ipm.org.br/an_ind_inaf_3.php, 2003>. Acesso em: out. 2008.

INSTITUTO PAULO MONTENEGRO. Inaf: 5^o indicador nacional de alfabetismo funcional. São Paulo: Instituto Paulo Montenegro, Ibope Opinião e ONG Ação Educativa. Disponível em: <http://www.ipm.org.br/an_ind_inaf_5. php, 2005>. Acesso em: out. 2008.

INSTITUTO PAULO MONTENEGRO. Indicador de alfabetismo funcional: Inaf/Brasil 2007. São Paulo: Instituto Paulo Montenegro, Ibope Opinião e ONG Ação Educativa. Disponível em: <http://www.ipm.org.br/download/ inaf06.pdf, 2007>. Acesso em: out. 2008.

ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT (OECD). Sample tasks from the PISA 2000 Assessment - Reading, mathematical, and scientific literacy. Paris, 2002a.

ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT (OECD). Reading for change: performance and engagement across countries. Paris, 2002b.

PROVA BRASIL. Disponível em: <http://provabrasil.inep.gov.br>. Acesso em: set. 2008.

PERFETTI, C. A. Sentences, individual differences, and multiple texts: three issues in text comprehension. Discourse Processes, v. 23, p. 337-355, 1997.

PRESSANTO, I. M. P. Material de leitura e desempenho leitor de universitários. CONGRESSO DE LEITURA DO BRASIL, 16, Anais... Campinas, São Paulo, 2007. Disponível em: <http://www.alb.com.br/anais16/sem12pdf/sm12ss12_ 01.pdf>. Acesso em: nov. 2008.

RIBEIRO, V. M. Por mais e melhores leitores: uma introdução. In: RIBEIRO, V. M. (Org.). Letramento no Brasil. São Paulo: Global, 2003.

ROSEMBLATT, L. M. The transactional theory of reading and writing. In: RRUDDEL, B.; RUDDELL, M. R.; SINGER, H. Theoretical models and processes of reading. Newark, DE: International Reading Association, 2004. p. 1363-1398.

SOARES, M. Literacy assessment and its implications for statistical measurement. Paris: UNESCO, 1992.

SOARES, M. Alfabetização e letramento. São Paulo: Contexto, 2003.

STAHL, S. A. Vocabulary and readability: How knowing word meanings affects comprehension. Top Language Disorders, v. 23, n. 3, p. 241-247, 2003.

STREET, B. V. Literacy in theory and practice. Cambridge: Cambridge University Press, 1984.

TRABASSO, T.; SUH, S.; PAYTON, P.; JAIN, R. Explanatory inferences and other strategies during comprehension and their effect on recall. In: LORCH, F.; O'BRIEN, E. (Ed.). Sources of coherence in reading. Hillsdale, NJ: LEA, 1995, p. 219-239.

UNICEF. Situação Mundial da Infância, 2003. Disponível em: <http://www.unicef.pt/docs/pdf_publicacoes/SOWC_2003.pdf>. Acesso em: set. 2008.

ZAKALUK, B. L.; SAMUELS, S. J. (Ed.). Readability: Its past, present, and future. International Reading Association: Newark, DE, 1988.

ZWAAN, R. A.; SINGER, M. Text comprehension. In: GRAESSER, C.; GERNSBACHER, M. A.; GOLDMAN, S. R. (Ed.). Handbook of Discourse Processes. Mahwah, NJ: LEA, 2003. p. 83-121.

Recebido em novembro de 2009.

Aprovado em dezembro de 2009.

¹

The authors would like to thank the two Brazilian institutions that partially financed this research: CAPES Foundation and Universidade do Oeste de Santa Catarina.

²

At the time these studies were conducted, primary school in Brazil included 8 years of schooling (ages 7-14). Currently, it includes 9 years (ages 6-14).

Publication Dates

• Publication in this collection
  03 Mar 2011
• Date of issue
  2010

History

• Received
  Nov 2009
• Accepted
  Dec 2009