Acessibilidade / Reportar erro

The evaluation of deaf people fluent in Brazilian Sign Language in the National High School Exam

Avaliação de pessoas surdas fluentes em Língua Brasileira de Sinais no Exame Nacional do Ensino Médio

Abstract

The present work analyzes the access of deaf people fluent in Brazilian Sign Language in the National High School Exam, which was created to measure the performance based on the curriculum and on the High School guidelines, which acquired the purpose of providing the access of High school seniors to higher education. The methodological theoretical approach upon which this study is based is the qualitative research of exploratory nature. The sequential mixed method was used for the collection and analysis of qualitative data and, based on these, the quantitative analysis was performed. The results show an increase in the number of deaf participants in the exams and justify the offer of bilingual exams to measure learning.

Keywords
Deafness; Educational measurement; Psychometrics; Sign language

Resumo

O presente trabalho analisa o acesso de pessoas surdas fluentes em Língua Brasileira de Sinais ao Exame Nacional do Ensino Médio. Este foi criado para mensurar o desempenho com base no currículo e nas diretrizes do ensino médio e adquiriu a finalidade de selecionar concluintes do ensino médio para ingresso em cursos superiores. A abordagem teórico-metodológica que fundamenta este trabalho é a pesquisa qualitativa de cunho exploratório. O método misto sequencial foi empregado para a coleta e a análise de dados qualitativos e, a partir destes, foi realizada a análise quantitativa. Os resultados evidenciam o aumento do número de participantes surdos nas provas e justificam a oferta de provas bilíngues para aferir a aprendizagem.

Palavras-chave
Surdez; Avaliação Educacional; Psicometria; Línguas de sinais

Access to education is a premise of the Federative Constitution of Brazil, available to Brazilian citizens of different religions, ethnicities, languages and social origins and the exams are part of the national education and evaluation system (Oliveira, 2015Oliveira, D. A. (2015). Nova gestão pública e governos democrático-populares: contradições entre a busca da eficiência e a ampliação do direito à educação. Educação e Sociedade, 36(132), 625-646. https://doi.org/10.1590/ES0101-73302015152440
https://doi.org/10.1590/ES0101-733020151...
). When available in an accessible format, such exams are designed to include people with different specificities and members of minority language communities, as mentioned by Silva (2015)Silva, I. R. (2015). Educação bilíngue para surdos e valorização de línguas minoritárias. Estudos Linguísticos, 44(2), 574-583. Recuperado de https://revistas.gel.org.br/estudos-linguisticos/article/view/995/577
https://revistas.gel.org.br/estudos-ling...
. In this sense, the video version of the Exame Nacional do Ensino Médio (Enem, National High School Exam) in Língua Brasileira de Sinais (Libras, Brazilian Sign Language) benefits people who are fluent in that language.

The first language of deaf people, Libras, was recognized as a means of communication and expression by Federal Law nº 10.436 of April 24, 2002. Enforced through Decree 5.626 of December 2005, the parameters for the provision of bilingual education have obtained visibility (Senna, 2019Senna, L. A. G. (2019). O Estatuto Linguístico da Língua Brasileira de Sinais e a superação do estigma na educação de surdos. Revista Brasileira de Educação Especial, 25(3), 487-https://doi.org/10.1590/s1413-65382519000300009
https://doi.org/10.1590/s1413-6538251900...
), such as the mandatory academic training of the Tradutor e Intérprete de Língua de Sinais (TILS, Sign Language Translator and Interpreter).

This profession was regulated by Law nº 12.319 of September 1, 2010, which established criteria for training in professional education, university extension, and continuing education courses. Proficiency certification was available until 2015, under the responsibility of public universities and their boards – made up of deaf teachers, linguists, and TILS with expertise in the subject.

This subject is directly linked to the theme of this article due to the fact that, among the competencies provided for in the exercise of this profession, is the support for accessibility and intervention in selection processes, tenders and exams. This dimension is also observed in the Brazilian Inclusion Law nº 13.146, of July 6, 2015 for the defense of equal rights aiming at social inclusion (Santos, 2016Santos, W. (2016). Deficiência como restrição de participação social: desafios para avaliação a partir da Lei Brasileira de Inclusão. Ciência e Saúde Coletiva, 21(10), 3007-3015. https://doi.org/10.1590/1413-812320152110.15262016
https://doi.org/10.1590/1413-81232015211...
) through the adequacy of the learning evaluation system. For this, he argues in favor of overcoming communication and technological barriers with a view to the full interaction of deaf people in Libras under the emphasis of bilingualism.

The defense of the discourse that bilingual education and the presence of TILS compose the list of linguistic rights of deaf individuals is a form of resistance, due to the historical process of implementing an inclusive school without the necessary planning and infrastructure. In this context is the fact that a good part of the deaf population has studied in inclusive schools, in processes of compulsory bilingualism (Silva, 2015Silva, I. R. (2015). Educação bilíngue para surdos e valorização de línguas minoritárias. Estudos Linguísticos, 44(2), 574-583. Recuperado de https://revistas.gel.org.br/estudos-linguisticos/article/view/995/577
https://revistas.gel.org.br/estudos-ling...
).

These data, qualitatively observed by linguists, are quantitatively identified in educational indicators. The statistical synopsis of the elementary education census (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira [Inep], 2019Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2019). Censo Escolar da Educação Básica 2018: notas estatísticas. Brasília: Autor. Recuperado de http://download.inep.gov.br/educacao_basica/censo_escolar/notas_estatisticas/2018/notas_estatisticas_censo_escolar_2018.pdf
http://download.inep.gov.br/educacao_bas...
) indicates that, of the 1,014.661 enrollments in inclusive regular classes, 20,893 of these are of deaf individuals, and 30,066 are hearing impaired individuals.

In addition to the perspective of the care provided by special education, it is worth noting the linguistic and ethnic characteristics of this population, since the accessibility policy must encompass the Brazilian cultural and linguistic diversity. In view of this, regarding ethnicity, 14,143 enrollments of black individuals, 133,159 of mixed-race individuals, and 1,514 indigenous individuals in inclusive regular classes were declared.

Although this group showed evidences of research and scientific work on the theme, attention to groups in different cultural and linguistic conditions for the dissemination of research in sign language linguistics is recent. With the contribution of studies by authors like Brito (2010)Brito, L. F. (2010). By a sign language grammar. Rio de Janeiro: Brazilian Weather., there was a reinterpretation of the historical process of constitution of linguistic communities. The research problem presented in this article is in this educational, linguistic, and cultural scenario.

The focus of this study is on the evaluation of deaf people who are fluent in Libras in the Enem exam. Created by ordinance nº 438 of the Ministry of Education, May 28, 1998, in 2009, it started to be used for the selection of admission to undergraduate courses in national and foreign universities through ordinance nº 109, of May 27, 2009. Currently, the results of the exam are used to award scholarships, to classify candidates in the Sistema de Seleção Unificada (Unified Selection System) and even to build parameters for job market placement. Given the relevance given to the performance obtained by the participant of the Enem, the choice for writing an article covering this topic is justified, focusing on the performance of deaf participants.

The Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep, National Institute for Educational Studies and Research Anísio Teixeira), which is responsible for the exam, provides a “Differentiated Service” for People with Disabilities, syndromes, global or specific disorders, etc. in order to guarantee accessibility (Inep, 2019Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2019). Censo Escolar da Educação Básica 2018: notas estatísticas. Brasília: Autor. Recuperado de http://download.inep.gov.br/educacao_basica/censo_escolar/notas_estatisticas/2018/notas_estatisticas_censo_escolar_2018.pdf
http://download.inep.gov.br/educacao_bas...
). The work of the specialized commission starts with the selection and adaptation of the exam items to the needs of people with visual impairment (for example, who have Braille exams at their disposal) and continues until the application of the exam.

Therefore, accessibility and inclusion issues permeate the entire process of preparing the exam and organizing its annual editions, including training the team that will apply the exam (Junqueira, Martins, & Lacerda, 2017Junqueira, R. D., Martins, D. A., & Lacerda, C. B. F. (2017). Política de acessibilidade e Exame Nacional do Ensino Médio (Enem). Educação e Sociedade, 38(139), 453-471. http://dx.doi.org/10.1590/es0101-733020171151513
https://doi.org/10.1590/es0101-733020171...
). In the midst of this vast scheme of operation, this study will stick to the public that has to take the exam in its linguistic uniqueness, being fluent in Libras.

The Enem is currently available in video format using Libras, however, until 2017, deaf participants had to read the question book, which was written in Portuguese Language (PL) – not in Libras –, and TILS were limited to clarifying specific doubts by participants. In the school routine, TILS must be in the classroom and their presence is foreseen in the National Policy on Special Education from the perspective of Inclusive Education (Silva & Fernandes, 2008Silva, D., & Fernandes, S. (2018). O Tradutor Intérprete De Língua De Sinais (TILS) e a política nacional de educação inclusiva em contextos bilíngues para surdos: um estudo da realidade da rede pública estadual paranaense. Revista Educação Especial, 31(60), 35-50. https://doi.org/10.5902/1984686X24814
https://doi.org/10.5902/1984686X24814...
).

In the performance of their duties, TILS interact with verbal and visual texts and discursively associate verbal and imagery elements providing the realization of intermodal translation (Quadros & Segala, 2015Quadros, R., & Segala, R. (2015). Tradução intermodal, intersemiótica e interlinguística de textos escritos em português para a Libras oral. Cadernos de Tradução, 35(2), 354-386. https://doi.org/10.5007/2175-7968.2015v35nesp2p354
https://doi.org/10.5007/2175-7968.2015v3...
), covering both oral and visual languages. Sometimes, it uses multimodal resources associating written Portuguese to image and gestural resources (Nicoloso & Herbele, 2015Nicoloso, S., & Heberle, V. (2015). As modalidades de tradução aplicadas à interpretação em língua de sinais brasileira. Cadernos de Tradução, 35(2), 197-235. https://doi.org/10.5007/2175-7968.2015v35nesp2p197
https://doi.org/10.5007/2175-7968.2015v3...
). The translation, as corroborated by Jakobson (2010, p.64, my translation)Jakobson, R. (2010). Linguística e comunicação. São Paulo: Cultrix. 2 2 In original: “intralingual, que consiste na reformulação ou interpretação dos signos verbais por meio de outros signos da mesma língua; a interlingual, definida como a tradução dos signos verbais de uma língua para os signos verbais de outra língua e a intersemiótica, a interpretação dos signos verbais por meio de sistemas de signos não-verbais” (Jakobson, 2010, p.64). , covers three types: “intralingual, which consists of the reformulation or interpretation of verbal signs by means of other signs of the same language; interlingual, defined as the translation of verbal signs from one language to verbal signs in another language, and intersemiotic, the interpretation of verbal signs through non-verbal sign systems”.

During translation, TILS perform semiotic and intersemiotic translation as a result of the difference in modality between oral and visual-spatial languages. Intralingual translation meets the diversity of students, when deaf people are in the same class, but at different times of language appropriation. Intersemiotic translation is constituted by the interface of systems of varied signs, means and codes: technology.

For this, the academic background and continuing education programs for TILS must be preserved in order to contribute to the education of the deaf and fully meet the linguistic rights of this group (Rodrigues & Beer, 2016Rodrigues, C. H., & Beer, H. (2016). Direitos, políticas e línguas: divergências e convergências na/da/para educação de surdos. Educação e Realidade, 41(3), 661-680. https://doi.org/10.1590/2175-623661114
https://doi.org/10.1590/2175-623661114...
). There are adversities in the evaluation of reading and text interpretation skills in both PL and foreign languages regarding deaf individuals who are fluent in Libras. The use of standardized methods to measure the mastery of the language (Travitzki, 2017Travitzki, R. (2017). Avaliação da qualidade do Enem 2009 e 2011 com técnicas psicométricas. Estudos em Avaliação Educacional, 28(67), 256-288. Recuperado de http://publicacoes.fcc.org.br/ojs/index.php/eae/article/view/3910
http://publicacoes.fcc.org.br/ojs/index....
) should include variations of both spoken and written language.

The concepts used in the preparation of the exam come from the High School (HS) curriculum and the exam itself is a construct (Travitzki, 2017Travitzki, R. (2017). Avaliação da qualidade do Enem 2009 e 2011 com técnicas psicométricas. Estudos em Avaliação Educacional, 28(67), 256-288. Recuperado de http://publicacoes.fcc.org.br/ojs/index.php/eae/article/view/3910
http://publicacoes.fcc.org.br/ojs/index....
), an instrument created to reach intrapsychic instances. As attested in the Inep (2015)Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2015). Relatório pedagógico: Enem 2011-2012. Brasília: Autor. Recuperado de http://portal.inep.gov.br/documents/186968/484421/Relat%C3%B3rio+Pedag%C3%B3gico+Enem+2011-2012/b29257e3-2a6c-44a3-992a-02130c379ba9?version=1.2
http://portal.inep.gov.br/documents/1869...
, the Item Response Theory allows for the collection of information and the comparison of results, with this being the only metric scale, while the qualitative and quantitative standards are predisposed in it.

From a statistical point of view, what is remarkable is the association of proficiency that increases according to the achievement of a higher position on this scale with the greater probability of getting the question right and the cumulative latent trait. This trait or aptitude corresponds to the psychosocial variables, elements that influence the subject during his or her response to the exam (Travitzki, 2017Travitzki, R. (2017). Avaliação da qualidade do Enem 2009 e 2011 com técnicas psicométricas. Estudos em Avaliação Educacional, 28(67), 256-288. Recuperado de http://publicacoes.fcc.org.br/ojs/index.php/eae/article/view/3910
http://publicacoes.fcc.org.br/ojs/index....
).

Deaf individuals, when subjected to the exam, have immediate contact with the label of the construct, in the Libras version of the exam, the translator’s body is seen, which, through their body movements and verbal-visual linguistic expressions, grants accessibility to the deaf who are fluent in Libras. The printed booklet of the objective questions of the exam presents the writing in PL and in English/Spanish, to evaluate their reading skills in a foreign language. In both ways, in the Libras version of the exam and in the PL version, the respondent’s skills and competencies (Hollas & Bernardi, 2020Hollas, J., & Bernardi, L. T. M. S. (2020). O Exame Nacional do Ensino Médio (Enem) e as competências para uma educação estatística crítica. Ensaio: Avaliação e Políticas Públicas em Educação, 28(106), 110-134. https://doi.org/10.1590/s0104-40362019002701489
https://doi.org/10.1590/s0104-4036201900...
) are measured through the choice of lexical items present in both the statement and the descriptors.

The participants’ answers provided the measurement of the discrimination, difficulty, and probability of randomly answering questions, a characteristic of the three-parameter logistic model, as stated by the Inep (2015)Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2015). Relatório pedagógico: Enem 2011-2012. Brasília: Autor. Recuperado de http://portal.inep.gov.br/documents/186968/484421/Relat%C3%B3rio+Pedag%C3%B3gico+Enem+2011-2012/b29257e3-2a6c-44a3-992a-02130c379ba9?version=1.2
http://portal.inep.gov.br/documents/1869...
. In addition to being able to measure what the person has learned about a concept, it will be discovered, based on the indicated descriptor, if the question was randomly answered or if it is more difficult than it should be due to the numerous deviations from the correct answer by the participants of the edition used as a test sample for the new (newly inserted) item.

The focus on skills and competencies developed during school years enables, according to the federal government, the monitoring of the improvements in the quality of public education. In 2010, minor adjustments were made to the content and the instrument and the reference matrix of the National Examination for the Certification of Youth and Adult Skills was adopted.

The scale has five competencies, common to all exams: (i) Mastering languages, (ii) Understanding phenomena, (iii) Facing problem situations, (iv) Build proposals, and (v) Developing proposals. For the composition of the evaluation instrument – an essay and four booklets with 45 multiple-choice questions –, the creators of the exam used the matrices of the Human Sciences and its technologies, Natural Sciences and its technologies, Mathematics and its technologies and, finally, Languages, Codes and their Technologies. For the written exam, there is a specific matrix with competencies and a set of knowledge objects associated with it.

The main reference for this path of construction of the indicators is the Law of Guidelines and Bases for National Education (Presidência da República, 1996Presidência da República (Brasil). (1996). Lei nº 9.394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da educação nacional. Brasília: Autor. Recuperado de http://www.planalto.gov.br/ccivil_03/LEIS/L9394.htm
http://www.planalto.gov.br/ccivil_03/LEI...
) which deals with the organization of the education system, providing for the mandatory common and regular public education in the national territory. Then, the Parâmetros Curriculares Nacionais (National Curriculum Parameters) and the Curricular Guidelines for High School Education weave a vast network of curricular content.

The existing matrices, in addition to being structured based on the Skills and Competencies Matrix of the National Examination of Competency Certification of Youth and Adults for High School, include other political debates of the period, such as the HS curricular reform and the curricular matrices of reference for the Evaluation System. Even so, each area evaluated in the Enem exam has knowledge objects associated with their respective reference matrices.

The “Language, codes and their technologies” area has nine competencies and includes the study of text, body practices, production and reception of artistic texts, study of literary and argumentative text, linguistic aspects in different texts, and the study of digital genres. Each area includes components of the regular curriculum base: History, Geography, Philosophy and Sociology, Chemistry, Physics and Biology, Mathematics and Portuguese Language, Literature, Foreign Language (English or Spanish), Arts, Education Physics, and Information Technologies. In relation to the production of the written text, the written exam is based on competencies that are:

... structural modalities of intelligence, or rather, actions and operations that we use to establish relationships with and between objects, situations, phenomena, and people we want to know. The skills derive from the acquired competencies and refer to the immediate “know-how” plan. Through actions and operations, skills are improved and articulated, enabling a new reorganization of competencies

(Inep, 2009Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2009). Matriz de Referência Enem. Brasília: Autor. Recuperado de http://portal.inep.gov.br/(http://download.inep.gov.br/educacao_basica/enem/downloads/2012/matriz_referencia_enem.pdf
http://portal.inep.gov.br/(http://downlo...
, p.15, my translation)3 3 In original: [...] modalidades estruturais da inteligência, ou melhor, ações e operações que utilizamos para estabelecer relações com e entre objetos, situações, fenômenos e pessoas que desejamos conhecer. As habilidades decorrem das competências adquiridas e referem-se ao plano imediato do “saber fazer”. Por meio das ações e operações, as habilidades aperfeiçoam-se e articulam-se, possibilitando nova reorganização das competências (Inep, 2009, p.15). .

Therefore, the relationship between competencies and skills is in the psychological instance that they represent. The reference matrices have as perspective the cognitive operation performed by the participant and the structural modalities of intelligence. Skills are branches of “competency standards” (Inep, 2009Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2009). Matriz de Referência Enem. Brasília: Autor. Recuperado de http://portal.inep.gov.br/(http://download.inep.gov.br/educacao_basica/enem/downloads/2012/matriz_referencia_enem.pdf
http://portal.inep.gov.br/(http://downlo...
, p.10, my translation)4 4 In original: ramificações da competência (Inep, 2009, p.10). , so the item is the materialization of skills and, therefore, requires action by the respondent.

In other words, through the Item Response Theory, with the aid of mathematical functions, it is possible to measure a person’s knowledge in relation to a reference content. Based on the response indicated by the participant, the evaluator will check his or her position on the scale of knowledge used for the construction of the construct. The calculation or statistical analysis becomes flexible to the peculiar aspects of the sample, that is, the mathematical function does not make a qualitative analysis inflexible, on the contrary, it relates the subject’s answer to a question and its performance in the other questions of a construct.

Thus, it is plausible that a construct such as the Enem is suitable for serving a group, because if the group obtains a negative performance, the construct should certainly be revised and adapted, which is beneficial to minority linguistic groups in view of their linguistic uniqueness. Therefore, statistical analysis covers the measurement of psychic processes also in complex sociolinguistic contexts.

Method

The theoretical and methodological approach underlying this study is a qualitative exploratory research (Minayo, 2017Minayo, M. C. S. (2017). Cientificidade, generalização e divulgação de estudos qualitativos. Ciência e Saúde Coletiva, 22(1), 16-17. https://doi.org/10.1590/1413-81232017221.30302016
https://doi.org/10.1590/1413-81232017221...
). The mixed sequential method, in accordance with Lorenzini (2017)Lorenzini, E. (2017). Pesquisa de métodos mistos nas ciências da saúde. Revista Cuidarte, 8(2), 1549-60. https://doi.org/10.15649/cuidarte.v8i2.406
https://doi.org/10.15649/cuidarte.v8i2.4...
and Oliveira, Magalhães, and MisueMatsuda (2018)Oliveira, J. L. C., Magalhães, A. M. M., & MisueMatsuda, L. (2018). Métodos mistos na pesquisa em enfermagem: possibilidades de aplicação à luz de Creswell. Texto e Contexto Enfermagem, 27(2), e0560017. https://doi.org/10.1590/0104-070720180000560017
https://doi.org/10.1590/0104-07072018000...
subsidized data collection and analysis.

The public documents available on the Inep website and which were analyzed in this study were: Official Paper nº 1, of June 18, 2010, Official Paper nº 7, of May 18, 2011, Official Paper nº 3, of May 24, 2012, Differentiated service at Enem, Item Response Theory, Statistical and educational data of Enem participants, Statistical and educational data of Enem participants, Higher education census 2010: technical summary, Higher education census: technical summary (2013), Reference Matrix of Mathematics and its technologies, Reference Matrix of Languages, codes, and their technologies, Reference Matrix of Sciences of nature and their technologies, Reference Matrix of human sciences and their technologies, and Reference Matrix of the Written Exam.

The convergence of knowledge disposed in the reference matrices to the definitions contained in the Base Nacional Comum Curricular (National Common Curricular Base) (Ministério da Educação, 2016Ministério da Educação (Brasil). (2016). Base Nacional Comum Curricular. Brasília: Autor. Recuperado de http://basenacionalcomum.mec.gov.br/
http://basenacionalcomum.mec.gov.br/...
) was noticeable, which, in 2017, integrated the national elementary education policy (Ferreti & Silva, 2017Ferreti, C. J., & Silva, M. R. (2017). Reforma do ensino médio no contexto da medida provisória n. 746/2016: Estado, currículo e disputas por hegemonia. Educação e Sociedade, 38(139), 385-404. https://doi.org/10.1590/es0101-73302017176607
https://doi.org/10.1590/es0101-733020171...
). The National Common Curricular Base composed the HS curriculum, serving as a reference for the formulation of curricula for the education systems of the States, the Federal District, and the municipalities. It discusses the curriculum defining competency as operated knowledge, that is, actions and operations used to establish connections between knowledge learned in different situations.

Inep’s open database has information on Enem since the first edition of the exam. The data segment analyzed was the request for resources and assistance in Libras and the performance of deaf people in the different areas of the Enem: Natural Sciences and their technologies, Human Sciences and their technologies, Mathematics, and Languages and codes; these data, in turn, were constituted by the extraction of microdata.

For the preparation of this article, the focus was on the stratification and identification of data related to the request for interpreters during the exam and/or other resources related to Libras, namely:

Translator-interpreter of Brazilian Sign Language (Libras): a professional with specific certification, qualified to mediate communication between deaf individuals and their listeners and, during the exam, clarify doubts of Libras users when reading words, expressions and sentences written in Portuguese. Translation of oral communications. Assist in understanding written texts. Provide synonyms and signs or words that help to understand the written word, idioms, sentences, and contexts.

Lip reading: reading service for people with hearing impairment (usually oral) who do not wish to communicate using Libras (Brazilian Sign Language), using interpretation techniques and reading lip movements. Guide-interpreter: a professional who is specialized in forms of communication and guiding, translation and interpretation techniques to mediate the interaction between people with deafblindness, the exam, and the others involved in the application of the National Exam

(Inep, 2015Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2015). Relatório pedagógico: Enem 2011-2012. Brasília: Autor. Recuperado de http://portal.inep.gov.br/documents/186968/484421/Relat%C3%B3rio+Pedag%C3%B3gico+Enem+2011-2012/b29257e3-2a6c-44a3-992a-02130c379ba9?version=1.2
http://portal.inep.gov.br/documents/1869...
, p.2, my translation)5 5 In original: Tradutor-intérprete de Língua Brasileira de Sinais (Libras): profissional com certificação específica, habilitado para mediar a comunicação entre surdos e ouvintes e, no ato da prova, esclarecer dúvidas dos usuários de Libras na leitura de palavras, expressões e orações escritas em Língua Portuguesa. Tradução das comunicações orais. Auxiliar na compreensão de textos escritos. Fornecer sinônimos e sinais ou palavras que ajudem a compreender a palavra escrita, as expressões idiomáticas, as orações e os contextos. Leitura labial: serviço de leitura da prova a pessoas com deficiência auditiva (geralmente oralizadas) que não desejam a comunicação por meio de Libras (Língua Brasileira de Sinais), valendo-se de técnicas de interpretação e da leitura dos movimentos labiais. Guia-intérprete: profissional especializado em formas de comunicação e técnicas de guia, tradução e interpretação para mediar a interação entre as pessoas com surdocegueira, a prova e os demais envolvidos na aplicação do Exame (Inep, 2015, p.2). .

The series of analyzed data refers to the participants who declared themselves as deaf or hearing impaired and who took the foreign language exam in written form, without the help of translators-interpreters (except in excerpts written in Portuguese). The performance in the exam, indicated by the grade obtained in it, was composed based on the exam format itself.

The interpretation of results was based on theoretical approaches, conducive to the study of factors that have serious implications for the social participation of linguistic minorities, which is reflected in the results of the evaluation. One point in evidence is the definition of linguistic capital as the opportunity that a person has to incorporate the linguistic resources of the society in which he or she participates (Bourdieu & Passeron, 2014Bourdieu, P., & Passeron, J. (2014). A reprodução. Rio de Janeiro: Francisco Alves.). When the linguistic majority is inaccessible in the visual sign modality, the deaf person has rare opportunities to expand the repertoire necessary to decipher social representations, appropriating this linguistic capital.

In this context, the researcher of the present study understands that the position occupied by TILS is that of a linguistic agent, a human resource that will facilitate the flow of linguistic exchanges between non-fluent and fluent Brazilian Sign Language speakers. Although, at times, the written discourse produced by deaf students does not meet the stylistic requirements already established by school standards, there is the possibility of negotiation and collective construction of a new way of seeing the other person and his or her language.

Thus, the written productions of those who have Portuguese as a second language will not necessarily follow the formal standards of the language for, as Labov (2008, p.313, my translation)Labov, W. (2008). Padrões sociolinguísticos. São Paulo: Parábola Editorial. 6 6 In original: “[são] traços da língua que caracterizam vários subgrupos numa sociedade heterogênea” Labov (2008, p.313). contributes, “[they are] features of the language that characterize various subgroups in a heterogeneous society”. Social and stylistic variation, for the author, assumes “saying ‘the same thing’ in several different ways, that is, the variants are identical in truth or referential value, but are opposed in their social and/or stylistic significance” (Labov, 2008Labov, W. (2008). Padrões sociolinguísticos. São Paulo: Parábola Editorial., p. 313, my translation)7 7 In original: “dizer ‘a mesma coisa’ de várias maneiras diferentes, isto é, as variantes são idênticas em valor de verdade ou referencial, mas se opõem em sua significação social e/ou estilística” (Labov, 2008, p. 313). .

It is possible to understand that, with Bourdieu, one could have a macrolinguistic analysis of linguistic relations, and in Labov, the quantitative description of the linguistic components relevant to the study of Enem is approached, due to the statistical aspects that guide it. Both theories contribute to the interpretation of the results of this study, above all, so a greater awareness of the complexity of the research problem can be achieved.

Results

The reading and analysis of the information presented in the pedagogical reports published by the Inep enabled the identification of an increased proportion of subscribers (Table 1). From these numbers, indications were observed due to the need for resources provided for in the Official Papers for each year. In this sense, it was found that the self-declaration of the hearing condition did not directly interfere in the choice of the participant for taking or not the Enem in Libras. The option for the presence of TILS on the day of the exam or the interpreter for lip reading was preserved as in the accessibility proposal prior to offering the video version of exam.

Table 1
Number of deaf participants in the Enem between 2010-2018

Oralized deaf people establish a more direct relationship with the spoken language, in social circulation, which can favor a greater mastery of vocabulary and idioms. This would be their linguistic “baggage”, which could be used by them in the performance of the Enem exam. Deaf individuals fluent in Libras who ask for an interpreter are entitled to the bilingual exam; this need was met with the Libras version of the objective evidence notebook (Figure 1).

Figure 1
Sign Language Translator and Interpreter and Lip Reading requests in the Enem exam between 2009-2018.

The complexity of this linguistic scenario requires extra care, especially when observing data on proficiency in the written exam, which has indicated the low performance of deaf people who are fluent in sign language. In competency 1, which evaluates the mastery of the standard cultured norm of the Portuguese language, there are demands regarding the formal use of the language, however, the grammatical deviations, common in texts produced by deaf individuals, may be being misinterpreted by those who correct the exam.

The democratic construction of a differentiated correction matrix for this public would provide a more in-depth analysis, the training of professionals for adequate correction and the careful monitoring of the results. The second competency of the written exam, the application of the concepts for the production of the argumentative written text allows to establish broad connections with the individual’s particular experience, since this moment requires a personal displacement of the subject in relation to the theme.

Participants who underwent the Translated video version of the exam received, in addition to the printed booklet with the questions, the kits containing their corresponding translations. This resource for specialized assistance was introduced as an experimental basis in 2017 and remained in the 2018 and 2019 editions. The interest in Libras by the individuals taking the exam also reflects on their performance. Among the deaf who opted for the Libras exam, 747 of them scored between 400 – 500 points, and 115 scored between 500 – 600 points. A smaller number were those who took the exam in the traditional version with the presence of the interpreter, but not with the exam in Libras.

The performance range for this group, made up of 501 participants, averaged 400 – 500 points; 41 deaf individuals scored between 500 – 600 points. Only three scored between 900 –1,000 points, but no one reached the maximum score, 1,000. Both cohesion and coherence are severely evaluated in competencies 3 and 4, whose focus is on the way the participant presents facts, pieces of information, and opinions connected to the defense of a point of view.

In addition, it is necessary to articulate the parts of the text to obtain a diversified repertoire of cohesive resources. To what extent would the deaf individual, who appropriates the written language in the current mold of the Brazilian education, be able to demonstrate an affinity with such competency? How has this knowledge been built in Brazilian schools in their varied inclusive, special, and bilingual contexts? The controversy in the different approaches to the teaching deaf individuals, characterized even in the current legislation, is the main element associated with the data of this research.

Discussion

The concept that permeates the studied exam is that the deaf person should appropriate, throughout the educational process, the reading and writing skills that are necessary to reading the questions and identifying the correct answers. The implications of the linguistic modality of sign language for writing should guide the criteria used for the elaboration of the correction matrix.

In fact, the consensus is limited as to what accepting represent in terms of the semantic plane, nor about the cultural aspects imbued in writing. The fact is that, in general, there is some damage to those who undergo tests and examinations, amidst all this complexity of elements that interfere in their educational process.

Like so many other groups of Brazilians who do not recognize themselves in the linguistic majority, there is the deaf person. Among educational trends, clinical and socio-anthropological approaches, they experience a social path marked by the movement of comings and goings, among concerns that are little understood and resolved.

The differentiated correction of written exams in Portuguese by deaf people who are fluent in Libras is not a new topic, on the contrary, it is the subject of old demands of the deaf community (Brito, 2010Brito, L. F. (2010). By a sign language grammar. Rio de Janeiro: Brazilian Weather.; Feneis, 2011Feneis. (2011, 26 de setembro). Carta denúncia: carta dos surdos falantes da língua brasileira de sinais ao Ministério Público Federal sobre a Política Nacional de Educação Especial na Perspectiva da Educação Inclusiva imposta à educação de surdos pela Secretaria de Alfabetização, Diversidade e Inclusão do Ministério da Educação [Vídeo]. Youtube. Recuperado de http://www.youtube.com/watch?v=m4j8OQhpxgA
http://www.youtube.com/watch?v=m4j8OQhpx...
; Fernandes & Moreira, 2017Fernandes, S., & Moreira, L. C. (2017). Políticas de educação bilíngue para estudantes surdos: contribuições ao letramento acadêmico no ensino superior. Educar em Revista, 3, 127-150. https://doi.org/10.1590/0104-4060.51048
https://doi.org/10.1590/0104-4060.51048...
). The defense based on the linguistic form of writing the exams, that is, with questions written in clearer terms and with an intelligible lexicon to this audience is available.

In the questions of the objective exam, the words and graphic-visual elements that contribute to the semantic interpretation are observed so that their meaning is constituted in a more comprehensive and supplementary context. In previous studies, Martins (2017)Martins, D. A. (2017). O Enem como via de acesso do surdo ao ensino superior brasileiro (Tese de doutorado não-publicada). Universidade Federal de São Carlos. and Martins and Lacerda (2015)Martins, D., & Lacerda, C. B. F. (2015). Exame Nacional do Ensino Médio e acesso de estudantes surdos ao Ensino Superior Brasileiro. Pro-Posições, 26(3), 83-101. http://dx.doi.org/10.1590/0103-7307201507804
https://doi.org/10.1590/0103-73072015078...
analyzed the data related to deaf participants in the Enem, revealing the low performance of this group.

In the written exam, the performance remains below average: of the 553 deaf individuals who took the exam, 211 scored between 200 and 300 points (out of a total of 1,000), and 179, between 300 and 400. Among the 721 students who took the video version of the exam, 289 scored between 200 – 300 points, and 224, between 300 and 400, with only five reaching 900 –1,000 points, but no one has reached 1,000.

Currently, the Diretoria de Avaliação da Educação Básica (Board of Elementary Education Assessment) is responsible for defining and proposing parameters, criteria, and mechanisms for conducting Elementary Education assessments. The Coordenação-Geral de Exames para Certificação (General Coordination for Certification Exams) is responsible for producing the instruments used to measure the proficiency in Libras (Inep, 2018Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2018). Microdados do Enem 2018. Brasília: Autor. Recuperado de http://inep.gov.br/microdados
http://inep.gov.br/microdados...
). At first, goals should be established to reverse the current results which, both in the short and long term, require serious interventions.

About Portuguese as a second language for deaf individuals, these guidelines should serve as a warning of possible errors in the correct use of canonical verbal inflection, of the infinitive form of the verb. Also, the low occurrence of connecting verbs, the frequent inflection of 1st person singular and plural, 3rd person singular, and present and past perfect tenses.

At the same time, with regard to reading comprehension, problems with the appropriation of the concept of punctuation and sequential cohesion can interfere with text interpretation. Although referential cohesion is preserved, when there are difficulties in appropriating the lexicon of a language, it is difficult to remain connected to the guiding thread of a written text.

As for the role of TILS and the guarantee of accessibility, it is possible to measure that its function is not limited to merely the function of a technical operational resource. The interpreter is a complex agent and his or her training is necessary in order to deal with the linguistic aspects of Libras and with the multimodality during the recording of the video version of the exam. The guiding principle of inclusion will thus be able to extrapolate the documents, improving the actions which have been implemented in recent years so that deaf individuals who are fluent in Libras have access to the Enem exam.

  • 2
    In original: “intralingual, que consiste na reformulação ou interpretação dos signos verbais por meio de outros signos da mesma língua; a interlingual, definida como a tradução dos signos verbais de uma língua para os signos verbais de outra língua e a intersemiótica, a interpretação dos signos verbais por meio de sistemas de signos não-verbais” (Jakobson, 2010Jakobson, R. (2010). Linguística e comunicação. São Paulo: Cultrix., p.64).
  • 3
    In original: [...] modalidades estruturais da inteligência, ou melhor, ações e operações que utilizamos para estabelecer relações com e entre objetos, situações, fenômenos e pessoas que desejamos conhecer. As habilidades decorrem das competências adquiridas e referem-se ao plano imediato do “saber fazer”. Por meio das ações e operações, as habilidades aperfeiçoam-se e articulam-se, possibilitando nova reorganização das competências (Inep, 2009Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2009). Matriz de Referência Enem. Brasília: Autor. Recuperado de http://portal.inep.gov.br/(http://download.inep.gov.br/educacao_basica/enem/downloads/2012/matriz_referencia_enem.pdf
    http://portal.inep.gov.br/(http://downlo...
    , p.15).
  • 4
    In original: ramificações da competência (Inep, 2009Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2009). Matriz de Referência Enem. Brasília: Autor. Recuperado de http://portal.inep.gov.br/(http://download.inep.gov.br/educacao_basica/enem/downloads/2012/matriz_referencia_enem.pdf
    http://portal.inep.gov.br/(http://downlo...
    , p.10).
  • 5
    In original: Tradutor-intérprete de Língua Brasileira de Sinais (Libras): profissional com certificação específica, habilitado para mediar a comunicação entre surdos e ouvintes e, no ato da prova, esclarecer dúvidas dos usuários de Libras na leitura de palavras, expressões e orações escritas em Língua Portuguesa. Tradução das comunicações orais. Auxiliar na compreensão de textos escritos. Fornecer sinônimos e sinais ou palavras que ajudem a compreender a palavra escrita, as expressões idiomáticas, as orações e os contextos. Leitura labial: serviço de leitura da prova a pessoas com deficiência auditiva (geralmente oralizadas) que não desejam a comunicação por meio de Libras (Língua Brasileira de Sinais), valendo-se de técnicas de interpretação e da leitura dos movimentos labiais. Guia-intérprete: profissional especializado em formas de comunicação e técnicas de guia, tradução e interpretação para mediar a interação entre as pessoas com surdocegueira, a prova e os demais envolvidos na aplicação do Exame (Inep, 2015Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2015). Relatório pedagógico: Enem 2011-2012. Brasília: Autor. Recuperado de http://portal.inep.gov.br/documents/186968/484421/Relat%C3%B3rio+Pedag%C3%B3gico+Enem+2011-2012/b29257e3-2a6c-44a3-992a-02130c379ba9?version=1.2
    http://portal.inep.gov.br/documents/1869...
    , p.2).
  • 6
    In original: “[são] traços da língua que caracterizam vários subgrupos numa sociedade heterogênea” Labov (2008, p.313)Labov, W. (2008). Padrões sociolinguísticos. São Paulo: Parábola Editorial..
  • 7
    In original: “dizer ‘a mesma coisa’ de várias maneiras diferentes, isto é, as variantes são idênticas em valor de verdade ou referencial, mas se opõem em sua significação social e/ou estilística” (Labov, 2008Labov, W. (2008). Padrões sociolinguísticos. São Paulo: Parábola Editorial., p. 313).

Acknowledgment

A. C. S. SOUZA, is a postdoctoral supervisor, contributes substantially to the expansion of data analysis in connection with studies in languages and sociolinguistics.

  • Support: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Programa Observatório de Educação) (n° 49/2012)
  • Article based on the doctoral dissertation of D.A. MARTINS, entitled “O Enem como via de acesso do surdo ao ensino superior brasileiro”. Universidade Federal de São Carlos, 2017.

How to cite this article

References

  • Bourdieu, P., & Passeron, J. (2014). A reprodução Rio de Janeiro: Francisco Alves.
  • Brito, L. F. (2010). By a sign language grammar Rio de Janeiro: Brazilian Weather.
  • Feneis. (2011, 26 de setembro). Carta denúncia: carta dos surdos falantes da língua brasileira de sinais ao Ministério Público Federal sobre a Política Nacional de Educação Especial na Perspectiva da Educação Inclusiva imposta à educação de surdos pela Secretaria de Alfabetização, Diversidade e Inclusão do Ministério da Educação [Vídeo]. Youtube. Recuperado de http://www.youtube.com/watch?v=m4j8OQhpxgA
    » http://www.youtube.com/watch?v=m4j8OQhpxgA
  • Fernandes, S., & Moreira, L. C. (2017). Políticas de educação bilíngue para estudantes surdos: contribuições ao letramento acadêmico no ensino superior. Educar em Revista, 3, 127-150. https://doi.org/10.1590/0104-4060.51048
    » https://doi.org/10.1590/0104-4060.51048
  • Ferreti, C. J., & Silva, M. R. (2017). Reforma do ensino médio no contexto da medida provisória n. 746/2016: Estado, currículo e disputas por hegemonia. Educação e Sociedade, 38(139), 385-404. https://doi.org/10.1590/es0101-73302017176607
    » https://doi.org/10.1590/es0101-73302017176607
  • Hollas, J., & Bernardi, L. T. M. S. (2020). O Exame Nacional do Ensino Médio (Enem) e as competências para uma educação estatística crítica. Ensaio: Avaliação e Políticas Públicas em Educação, 28(106), 110-134. https://doi.org/10.1590/s0104-40362019002701489
    » https://doi.org/10.1590/s0104-40362019002701489
  • Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2009). Matriz de Referência Enem Brasília: Autor. Recuperado de http://portal.inep.gov.br/(http://download.inep.gov.br/educacao_basica/enem/downloads/2012/matriz_referencia_enem.pdf
    » http://portal.inep.gov.br/(http://download.inep.gov.br/educacao_basica/enem/downloads/2012/matriz_referencia_enem.pdf
  • Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2015). Relatório pedagógico: Enem 2011-2012 Brasília: Autor. Recuperado de http://portal.inep.gov.br/documents/186968/484421/Relat%C3%B3rio+Pedag%C3%B3gico+Enem+2011-2012/b29257e3-2a6c-44a3-992a-02130c379ba9?version=1.2
    » http://portal.inep.gov.br/documents/186968/484421/Relat%C3%B3rio+Pedag%C3%B3gico+Enem+2011-2012/b29257e3-2a6c-44a3-992a-02130c379ba9?version=1.2
  • Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2018). Microdados do Enem 2018 Brasília: Autor. Recuperado de http://inep.gov.br/microdados
    » http://inep.gov.br/microdados
  • Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. (2019). Censo Escolar da Educação Básica 2018: notas estatísticas Brasília: Autor. Recuperado de http://download.inep.gov.br/educacao_basica/censo_escolar/notas_estatisticas/2018/notas_estatisticas_censo_escolar_2018.pdf
    » http://download.inep.gov.br/educacao_basica/censo_escolar/notas_estatisticas/2018/notas_estatisticas_censo_escolar_2018.pdf
  • Jakobson, R. (2010). Linguística e comunicação São Paulo: Cultrix.
  • Junqueira, R. D., Martins, D. A., & Lacerda, C. B. F. (2017). Política de acessibilidade e Exame Nacional do Ensino Médio (Enem). Educação e Sociedade, 38(139), 453-471. http://dx.doi.org/10.1590/es0101-733020171151513
    » https://doi.org/10.1590/es0101-733020171151513
  • Labov, W. (2008). Padrões sociolinguísticos São Paulo: Parábola Editorial.
  • Lorenzini, E. (2017). Pesquisa de métodos mistos nas ciências da saúde. Revista Cuidarte, 8(2), 1549-60. https://doi.org/10.15649/cuidarte.v8i2.406
    » https://doi.org/10.15649/cuidarte.v8i2.406
  • Martins, D. A. (2017). O Enem como via de acesso do surdo ao ensino superior brasileiro (Tese de doutorado não-publicada). Universidade Federal de São Carlos.
  • Martins, D., & Lacerda, C. B. F. (2015). Exame Nacional do Ensino Médio e acesso de estudantes surdos ao Ensino Superior Brasileiro. Pro-Posições, 26(3), 83-101. http://dx.doi.org/10.1590/0103-7307201507804
    » https://doi.org/10.1590/0103-7307201507804
  • Minayo, M. C. S. (2017). Cientificidade, generalização e divulgação de estudos qualitativos. Ciência e Saúde Coletiva, 22(1), 16-17. https://doi.org/10.1590/1413-81232017221.30302016
    » https://doi.org/10.1590/1413-81232017221.30302016
  • Ministério da Educação (Brasil). (2016). Base Nacional Comum Curricular Brasília: Autor. Recuperado de http://basenacionalcomum.mec.gov.br/
    » http://basenacionalcomum.mec.gov.br/
  • Nicoloso, S., & Heberle, V. (2015). As modalidades de tradução aplicadas à interpretação em língua de sinais brasileira. Cadernos de Tradução, 35(2), 197-235. https://doi.org/10.5007/2175-7968.2015v35nesp2p197
    » https://doi.org/10.5007/2175-7968.2015v35nesp2p197
  • Oliveira, D. A. (2015). Nova gestão pública e governos democrático-populares: contradições entre a busca da eficiência e a ampliação do direito à educação. Educação e Sociedade, 36(132), 625-646. https://doi.org/10.1590/ES0101-73302015152440
    » https://doi.org/10.1590/ES0101-73302015152440
  • Oliveira, J. L. C., Magalhães, A. M. M., & MisueMatsuda, L. (2018). Métodos mistos na pesquisa em enfermagem: possibilidades de aplicação à luz de Creswell. Texto e Contexto Enfermagem, 27(2), e0560017. https://doi.org/10.1590/0104-070720180000560017
    » https://doi.org/10.1590/0104-070720180000560017
  • Presidência da República (Brasil). (1996). Lei nº 9.394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da educação nacional Brasília: Autor. Recuperado de http://www.planalto.gov.br/ccivil_03/LEIS/L9394.htm
    » http://www.planalto.gov.br/ccivil_03/LEIS/L9394.htm
  • Quadros, R., & Segala, R. (2015). Tradução intermodal, intersemiótica e interlinguística de textos escritos em português para a Libras oral. Cadernos de Tradução, 35(2), 354-386. https://doi.org/10.5007/2175-7968.2015v35nesp2p354
    » https://doi.org/10.5007/2175-7968.2015v35nesp2p354
  • Rodrigues, C. H., & Beer, H. (2016). Direitos, políticas e línguas: divergências e convergências na/da/para educação de surdos. Educação e Realidade, 41(3), 661-680. https://doi.org/10.1590/2175-623661114
    » https://doi.org/10.1590/2175-623661114
  • Santos, W. (2016). Deficiência como restrição de participação social: desafios para avaliação a partir da Lei Brasileira de Inclusão. Ciência e Saúde Coletiva, 21(10), 3007-3015. https://doi.org/10.1590/1413-812320152110.15262016
    » https://doi.org/10.1590/1413-812320152110.15262016
  • Senna, L. A. G. (2019). O Estatuto Linguístico da Língua Brasileira de Sinais e a superação do estigma na educação de surdos. Revista Brasileira de Educação Especial, 25(3), 487-https://doi.org/10.1590/s1413-65382519000300009
    » https://doi.org/10.1590/s1413-65382519000300009
  • Silva, I. R. (2015). Educação bilíngue para surdos e valorização de línguas minoritárias. Estudos Linguísticos, 44(2), 574-583. Recuperado de https://revistas.gel.org.br/estudos-linguisticos/article/view/995/577
    » https://revistas.gel.org.br/estudos-linguisticos/article/view/995/577
  • Silva, D., & Fernandes, S. (2018). O Tradutor Intérprete De Língua De Sinais (TILS) e a política nacional de educação inclusiva em contextos bilíngues para surdos: um estudo da realidade da rede pública estadual paranaense. Revista Educação Especial, 31(60), 35-50. https://doi.org/10.5902/1984686X24814
    » https://doi.org/10.5902/1984686X24814
  • Travitzki, R. (2017). Avaliação da qualidade do Enem 2009 e 2011 com técnicas psicométricas. Estudos em Avaliação Educacional, 28(67), 256-288. Recuperado de http://publicacoes.fcc.org.br/ojs/index.php/eae/article/view/3910
    » http://publicacoes.fcc.org.br/ojs/index.php/eae/article/view/3910

Publication Dates

  • Publication in this collection
    28 Aug 2020
  • Date of issue
    2021

History

  • Received
    11 Nov 2019
  • Reviewed
    24 Mar 2020
  • Accepted
    30 June 2020
Programa de Pós-Graduação em Psicologia, Pontifícia Universidade Católica de Campinas Núcleo de Editoração SBI - Campus II, Av. John Boyd Dunlop, s/n. Prédio de Odontologia, 13060-900 Campinas - São Paulo Brasil, Tel./Fax: +55 19 3343-7223 - Campinas - SP - Brazil
E-mail: psychologicalstudies@puc-campinas.edu.br