Accessible Enem: Autonomy for the Person with Total Visual Impairment in the Brazilian High School National Examination²

ABSTRACT:

Currently, for the visually impaired person to complete the Brazilian High School National Examination (called ENEM), adapted material and human assistance are available, the latter with the purpose of reading the test, template filling and piece of writing transcription. Consequently, the disabled person depends heavily on other people for completing the test. In this sense, the objective of this study is to identify the accessibility issues that exist in the current ENEM process and to present a solution that, through an application with digital accessibility called Accessible ENEM, enables the participation of the person with total visual impairment with autonomy in the examination. Based on the User Centered Design methodology and qualitative survey research, the application has been specified, constructed, and evaluated. The results of the study, which involved the participation of 33 people with total visual impairment in its various stages, demonstrate that the existing communication barriers may be eliminated with the use of the computer and the Accessible ENEM application, allowing people with visual impairment to complete the Higher Education entrance exam with autonomy and in equal conditions in relation to the other participants.

KEYWORDS:
Special Education; Blindness; High School National Examination

RESUMO:

atualmente, para pessoa com deficiência visual total realizar o Exame Nacional do Ensino Médio (ENEM), disponibilizam-se materiais adaptados e auxílio humano, este com a finalidade de leitura da prova, preenchimento de gabaritos e transcrição da redação. Consequentemente, para a realização da prova, a pessoa com deficiência depende fortemente de outras pessoas. O objetivo deste estudo é identificar as questões de acessibilidade existentes no processo atual do ENEM e apresentar uma solução que, por meio de um aplicativo com acessibilidade digital denominado ENEM Acessível, viabiliza a participação da pessoa com deficiência visual total com autonomia no exame. Com base na metodologia de Projeto Centrado no Usuário e em pesquisa de levantamento qualitativa, o aplicativo foi especificado, construído e avaliado. Os resultados do estudo, que contou com a participação de 33 pessoas com deficiência visual total em suas várias etapas, demonstram que as barreiras de comunicação atualmente existentes podem ser eliminadas com o uso do computador e do aplicativo ENEM Acessível, permitindo que pessoas com deficiência visual total realizem a prova de ingresso no Ensino Superior com autonomia e em condições de igualdade com os demais participantes.

PALAVRAS-CHAVE:
Educação Especial; Cegueira; Exame Nacional do Ensino Médio

1 Introduction

Technological evolution has brought about profound changes in the form of access to education and information and in relationships among people. Likewise, the growing use of computers and assistive technology has enabled new possibilities for autonomy and social inclusion of people with disabilities (Torres, Mazzoni, & Alves, 2002; Secretaria Especial dos Direitos Humanos, 2009; Galvão, 2009). The term Assistive Technology (AT) is used to denote '[...] resources and services that contribute to providing or enhancing functional abilities of people with disabilities' (Bersch, 2017, p. 2).

The legal definition of visual impairment in Brazil identifies two types: blindness, or total visual impairment, and low vision. According to Decree-Law No. 5,296 (2004), blindness is the condition of visual acuity equal to or less than 0.05 in the best eye, with the best optical correction. In this paper, the focus is on people with this condition.

Currently, although the visually impaired person frequently uses computers and cell phones with the 'Screen Reader' AT, which converts texts into voice messages in their daily activities, these resources are not regularly made available during that person's participation in the university selective processes for various reasons. This makes to carry out the assessment with full autonomy on the part of the person with total visual impairment difficult.

The person with disability is guaranteed by law the right to make use of reasonable adaptations necessary to ensure equal opportunities in all situations. Regarding Higher Education, Decree-Law No. 6,949 of 2009, in its Article 24 (Education), provides that people with disabilities '[...] may have access to Higher Education in general, [...] without discrimination and on an equal footing. To this end, States Parties shall ensure the provision of reasonable adaptations for people with disabilities' (Decreto n º 6.949, 2009). Subsequently, the Brazilian Law of Inclusion – Decree-Law No. 13,146 of 2015 - was more explicit and determined that, for selective processes in courses offered by educational institutions, 'adequate accessibility and assistive technology resources' should be made available, 'previously requested and chosen by the disabled candidate' (Decreto n º 13.146, 2015).

The enrollment of people with disabilities in Higher Education has increased significantly in recent years (Silva & Meletti, 2014; Santos, 2011; Martins, Leite, & Lacerda, 2015). However, the proportion of people with disabilities in the totality of students of Higher Education is still lower than its proportion in the population, with indices lower than 1% in relation to the vacancies offered (Duarte, Rafael, Filgueiras, Neves, & Ferreira, 2013).

Selective university processes, such as tests and exams, are methods of evaluation of learning and objects of study of the sub-area of 'teaching-learning' in the area of education knowledge (Coordination for the Improvement of Higher Education Personnel [CAPES], 2014). In this context, currently, ENEM is the university selection process with the largest number of participants in Brazil. The 2016 edition of this exam totalled to more than nine million enrollees. Among them, 72,093 requested accessibility resources, of which 17,433 presented issues related to visual impairment including 1,414 with total visual impairment (Ministério da Educação, 2016). ENEM grade is adopted in the university selection by the majority of the public Institutes of Higher Education (IHS) and by some of the private IHS. It is also used to attain scholarships for technical courses, through the National Program for Access to Technical Education and Employment (called PRONATEC) (Ministério da Educação, 2017a; Ministério da Educação, 2017b). Due to the importance of ENEM, this study adopted it as a selective process model for the analysis of accessibility to the person with total visual impairment.

Although the right to reasonable adaptations is ensured by law in university selection processes and considering there is already international experience with large-scale assessments that provide computer and AT for the total visually impaired (Act, 2016; College board, 2017), ENEM is not ready to allow its use in the examination (Ministério da Educação, 2012a). Currently, in order for a person with total visual impairment to be able to carry out ENEM, the test in Braille and human assistance are available in two types of specialized services: reader⁷ and transcriber⁸ assistances (Ministério da Educação, 2012a). The need for these services links the performance of the test by the person with a disability to the abilities and qualification of other people, who are subject to human flaws, which may affect the performance of the candidate during the test (Fujiyoshi, Fujiyoshi, & Aomatsu, 2010; Sanchez, Espinoza, & Campos, 2014).

This paper reports part of the master's research entitled Acessibilidade digital em processos seletivos universitários para a pessoa com deficiência visual: desenvolvimento do aplicativo ENEM Acessível (Digital accessibility in university selective processes for the visually impaired: development of the Accessible ENEM application) (Leria, 2016). This research specified, developed and evaluated an application for the total visually impaired person to perform the examination with autonomy, which is conventionally called Accessible ENEM. The research was approved by the research ethics committee of the Universidade Federal do ABC (UFABC) (CAAE no. 37334514.0.0000.5594).

The focus of this research report is to highlight the issues of accessibility and the difficulties faced due to the lack of autonomy in the current evaluation process, considering the resources made available to ENEM. In addition, we aim to show how Accessible ENEM application project solved the problems identified, allowing the person with total visual impairment to perform the test with autonomy.

In this way, this study presents, in Section 2, the methodology used. Then, in Section 3, the results obtained are shown as follows: the current ENEM scenario for the total visually impaired person, the accessibility barriers identified and the resources developed in the Accessible ENEM application. In Section 4, we discuss how the barriers were eliminated or minimized by the use of the application, and finally in Section 5, we present the conclusions of the research.

2 Method

According to the taxonomy of Wazlawick (2008), in his book Metodologia de pesquisa para Ciência da Computação (Research methodology for Computer Science), the research carried out was considered as exploratory of the 'presentation of something different' style, since it presents a different way of solving a problem, using computational technology. This type of research is based on the qualitative analysis of information. The study is of applied nature, with an exploratory and descriptive objective (Wazlawick, 2008).

The methodological framework used to generate an application within the guidelines of digital accessibility and the autonomy of the person with disability in ENEM is the User Centered Design⁹. This methodology has the 'focus on the user' principle, which stipulates that the project must meet the users' needs and help them achieve their goals. According to Barbosa and Silva (2017, p. 115)

The sooner users are involved in the design process [...], the sooner it will be possible to learn about their needs and thus positively influence the synthesis of the solution, as well as identify and correct any problems.

Thus, the research and development of Accessible ENEM counted on the collaboration of the person with disability in all its phases, from the survey of needs, specification and construction to the final phase of evaluation of the application.

The stages of the research project, briefly described in the following items, were the bibliographic review, the identification of the resources currently offered by ENEM for the person with disability, the survey of the needs in order to adapt the exam, the development of the application and the tests of validation for people with disabilities.

2.1 Bibliographic review

The literature analyzed for the study was divided into three distinct lines of research: the participation of people with disabilities in Higher Education to evaluate the impact of the research; the public policies of attention to the person with disability, to prove the right of people with disabilities to the intervention proposed by the project; and the theme of digital accessibility and AT in Education, with the objective of identifying project requirements for the application.

2.2 Identification of ENEM current resources for the person with disability

In order to obtain the formats of the tests offered by ENEM and their respective accessibility resources, four editions of the exam were analyzed, available on the website of the National Institute of Pedagogical Studies Anísio Teixeira (called INEP) (Ministério da Educação, 2015), the Nota Técnica - Atendimento Diferenciado no ENEM (Technical Note - Distinguished Service in ENEM) (Ministério da Educação, 2012a) and the Edital do ENEM 2016 (ENEM Official Publication) (Edital nº 10, 2016). From the participation data in the exam released by INEP, the resources effectively used by people with total visual impairment were identified in these exam editions (Ministério da Educação, 2012b).

2.3 Survey of the needs for the exam adaptation

According to the User Centered Design methodology, the potential user of the system is given voice. In order to obtain the user requirements for the Accessible ENEM application, the Accessibility Survey for the visually impaired people in the Brazilian High School National Examination was carried out, which had the participation of 28 people with disabilities. The objective was to identify the experience of the total visually impaired person with the accessibility resources currently available to ENEM.

Its application was made through a questionnaire published and completed on the internet with the use of the tool Google Forms, a free tool that allows the preparation of forms accessible to the person with total visual impairment. The accessibility of the research form was evaluated by two people with total visual impairment, prior to the dissemination of the research. The answer alternative in Word form was also offered for access.

The criteria for a subject to be included in the study were to have participated in the ENEM test, to have taken the examination with human assistance of the reader and to be a person with visual impairment. It should be noted that the candidate who requests Braille is also assisted by the reader/transcriber. It was decided to select participants who had already taken the test so that they could report the individual experience and inform the positive and negative points of the resources made available by INEP for the person with disability.

The strategies used for dissemination included contact with educational institutions, associations of people with disabilities and government agencies, sending emails, interviews on radio stations, dissemination on websites and social network.

2.4 Development and application tests

Based on the User Centered Design methodology that expects prototyping and testing cycles, a first version of the Accessible ENEM application was developed. This version was improved through the analyzes of the interaction of people with total visual impairment with the application. This stage of the research is described in the article ENEM Acessível: Desenvolvimento de aplicativo para participação autônoma da pessoa com deficiência visual total (Accessible ENEM: Application development for autonomous participation of the person with total visual impairment) (Leria, Ferreira, & Fraga, 2016).

The evaluation of the final version of the application was carried out by three volunteers with disabilities, different from those who participated in the development, in order to evaluate the usability and the autonomy in the use of the application.

3 Results

This section brings the relevant research results to the focus of this article.

3.1 Legislation of attention to the person with disability in Education

Table 1 is the result of the bibliographic review, which identified the legislations related to accessibility as a right of the person with disability in university selective processes.

Table 1
Publications analyzed in the research

Publications / Sources	Contributions
Decree No. 6.949(2009)	It presents the concept of disability and accessibility as a right of the person with disability.
Decree No. 5.296 (2004)	It defines visual impairment and legitimizes accessibility in the area of communication and information.
Decree No. 13.146(2015)	It ensures the use of the computer and AT in university selective processes.
Decree No. 3.298(1999)	It regulates Special Education in Higher Education.
Law No. 9.394(1996)	It ensures the person with disability the right to attend common classes.
Constitution (Constituição da República Federativa do Brasil, 1988)	It defines discrimination on the grounds of disability and recognizes lawful forms of positive discrimination.
Technical Assistance Committee(Secretaria Especial dos Direitos Humanos, 2009)	It defines AT.

Source: Elaborated by the authors.

3.2 Current scenario of people with disabilities in ENEM

The resources provided by INEP for candidates with disabilities are summarized below. Based on the information presented in the Nota Técnica - Atendimento Diferenciado no ENEM (Technical Note - Distinguished Service in ENEM) (Ministério da Educação, 2012a), the following resources are made available for the person with disability during the exam: test in Braille, reader assistance, test of the reader¹⁰ transcription assistance, individual room, 1 hour additional time and interpreter guide.

The ENEM selection process takes place over two days of exams, of different durations that evaluate different areas of knowledge. Each edition of the exam has one piece of writing and 180 multiple choice questions, 90 on each test day, and are divided into groups of 45 questions, covering the four areas of knowledge: 1 - Human Sciences and their Technologies; 2 - Natural Sciences and their Technologies; 3 - Languages, Codes and their Technologies and 4 - Mathematics and its Technologies.

According to the ENEM Official Publication 2016 (Edital nº 10, 2016), people with disabilities may request an additional hour to finish the exam. Thus, the person with disability has an average response time of three minutes and 40 seconds per question, which adds 40 seconds per question in relation to the other participants.

In the exam editions analyzed - those from 2011 to 2014 - the writing test question presents the following format to describe the proposed theme (Ministério da Educação, 2015): an introductory text; texts and/or images that describe the proposed theme, which can vary from one to three contents depending on the test; and test instructions. According to INEP guidelines, the writing test question must respect the minimum limit of 7 lines and maximum of 30 lines of content. INEP also advises that the answers of multiple-choice questions and the piece of writing should be transcribed into their respective response cards and writing sheets (Edital nº 10, 2016). It was observed that an edition of the exam presents an average of 100 images in addition to the great volume of text, containing about 30 pages per exam in ink.

3.3 barriers and adequacy needs for the person with total visual impairment

In this section, we detail the views of people who answered the Accessibility Survey for the visually impaired people in the High School National Examination about how the resources identified in the previous section are effectively made available and used. The description of the resources are shown in Table 2.

Table 2
Accessibility resources for the person with total visual impairment in ENEM

Resource	Description of resource use
Test in Braille	The exam is transcribed in Braille (text, mathematical formulas and drawings of simple figures). A private room, two readers and the 'Test of the reader', which is a test in ink with descriptions of the images, are made available with the test in Braille. Although the 'Braille Machine'1 does not appear in the ENEM Official Publication as an available resource, the research reports indicate that four people used this resource for the writing test during the ENEM exam in 2010, 2013 and 2015, in the states of Minas Gerais, São Paulo and Rio de Janeiro. The participant is placed in a private room and receives the test in Braille with the drawings of the figures in high relief. In addition, he/she receives assistance of readers to help in reading, if they are requested, and to understand the images, whose descriptions are only in the 'Test of the reader'. The readers also act as transcribers to fill out the response card and transcribe the piece of writing by using ink. For transcription the participant must dictate the writing, spelling the accent marks, punctuation and special characters.
Reader assistance	Specialized reading service for people with disability. The readers act in pairs and have the 'Test of the reader'. The participant is placed in a private room and receives the assistance of readers to read the test and description of images. The readers alternate when reading. The 'Test of the reader' contains the description of illustrations, images, maps, tables and graphs. The readers also act as transcribers, as described in the previous item (Test in Braille).
Transcriptionassistance	Specialized service to fill out multiple choice and complete writing questions evidence for participants unable to write or fill in the response card. The participant is placed in a private room and receives the assistance of the transcribers to fill out the response card and transcribe the piece of writing by using ink. The transcribers act in pairs, often in partnership with the readers, as described in the item 'Test in Braille'. The transcriber does not receive the test of the reader.
Interpreter guide	A professional specialized in forms of communication and techniques of guidance, translation and interpretation to mediate the interaction between people with deafness/blindness, the test and the others involved in the application of the exam. Not enough information was found to describe the performance of the support of the guide-interpreter for deaf-blind people, in addition to the fact that the resource is applied in a private room.

Source: Elaborated by the authors.
• ¹
  'Braille Machine': A typewriter in Braille, which allows typing the text in Braille and prints in Braille and in Portuguese with the writing in ink.

In order to systematize the reports obtained in the research, the act of performing the test as a process containing four activities was decided, namely: 1- access the content of the test; 2 - understand the information; 3 - solve the question - activity that depends only on the student; and 4 - demonstrate knowledge by marking the alternative or writing the essay. Based on this process, during the analysis of the research reports, barriers to take the ENEM were identified in three of these four activities.

3.3.1 Barriers to content access of the exam

Regarding the barriers to content access, the testimonies reported difficulties both for those who ask for the assistance of the reader and for those who request the test in Braille. Considering the test in Braille, the large number of texts generates a large volume of printed sheets (three times the amount of the test in ink), which makes handling difficult. The large volume of the tests in Braille and the short amount of time to answer the questions were reported as causes of impediment of the complete reading of the test. It is important to reiterate that visually impaired people have different levels of Braille proficiency, but on average '[...] Braille readers read at about half the speed of print readers' (Pring, 1984 as cited in Dimitrova-Radojichikj, 2015, p. 2).

For those who use the human assistance of the reader, there were flaws in the reader's training for reading, especially specific terms such as chemical and mathematical formulas, as well as reading texts in other languages or even the fluent reading of Portuguese.

3.3.2 Barriers in the understanding of information

According to reports, the understanding of the text is hindered, because the reading (spoken) by the reader is not the habitual way, practiced in everyday life of the people with total visual impairment. In addition, the lack of autonomy to re-read the text, a necessary process for understanding, creates embarrassment, fatigue and discomfort.

The readers' alternate during the test, a practice used to avoid the fatigue of these professionals, and the consequent alteration in the form of reading and speech speed, requires adaptation and resulting cognitive effort on the part of the person with total visual impairment. The lack of autonomy to read and re-read the sentences leads to a greater cognitive effort, due to the need to change focus (from the understanding of the text) to communication with the reader (verbalize what should be read again) requesting to return to the previous sentence, repeat the last word or read a paragraph again. Thus, besides the reported embarrassment, the process for understanding a spoken text, which, at first, seems simple, requires an additional effort from those with total visual impairment.

Another barrier that hinders the understanding of the content is the amount of images in the exam (average of 100 images per edition), that is, they account for approximately 50% of the content of the questions. Although the images are described, many are complex and the descriptions are not sufficient to allow their understanding. The additional effort to understand the texts (spoken), considering the length of the exam (5 hours on the first day and 6 hours and 30 minutes on the second day), in addition to the image descriptions that are difficult for the person with total visual impairment to understand. Although a comparative study of the cognitive load of the candidate with total visual impairment has not been done in relation to the non-disabled candidate, we can state that the burden is higher and that the person with disability is at a disadvantage due to the loss of autonomy.

3.3.3 Barriers to express knowledge

In order to complete the writing test, the person with total visual impairment has to mentally elaborate the sentence, dictate the text to the reader, including punctuation marks, accent marks and the spelling of complex words and request proofreading and transcription of the text several times. It is also necessary to control the number of lines in the piece of writing. According to the research, the lack of autonomy in this process hinders its result, generating lack of confidence in the transcription and constraint, as well as time wasting.

In addition, the transcription of the responses on the response card and the possibility of failures in this process also generate lack of confidence and discomfort in relation to the test. According to the participants, at the end of the test, the reader is also tired and subject to failure. The mental effort to write and dictate the text, besides the lack of autonomy, generate conditions of inequality for the visually impaired to perform the test.

3.4 Accessible ENEM application and its validation

Based on the survey of accessibility barriers, the functionality of Accessible ENEM has been specified and developed. The Accessible ENEM was designed according to the concept of Universal Design1¹¹. For this reason, it has a graphic interface and can be used by people with or without disabilities.

Accessibility in digital environments enables access to any type of digital content for the user, regardless of their disability. For this to be possible in the web environment, the application must be built according to the World Wide Web Consortium (W3C) guidelines, which regulates content interpretation standards and application development with web accessibility rules. In the development of Accessible ENEM, besides the W3C accessibility recommendations (Web Content Accessibility Guidelines [WCAG2], 2008), the ISO-IEC 25010 (International Organization for Standardization, & International Electrotechnical Commission, 2011) quality criteria were also used to specify and evaluate the application.

The Accessible ENEM works seamlessly with screen reader technology, which makes use of speech synthesis technology, turning texts into an audible response. For the study, the use of AT NVDA (NV Access, 2015) was chosen because it is a free, good quality software and well accepted by people with total visual impairment (W3C-Brasil, 2013).

The functions of the application are divided into four distinct modules: identification, test questions, writing test and finalization of the test. The user starts navigating the Accessible ENEM by reading the instructions page, which presents the features of the application, then the user inputs the data for identification on the exam and starts the test. From there, he/she answers the questions in the multiple choice questions module, writes the essay, and ends the test.

It is important to note that the candidate does not need to know the various features of the NVDA Screen Reader to use the application. The design of the application took into account that not all candidates will be familiar with screen readers, therefore this knowledge could not be taken into account for the use of the application, hence, navigating around the various modules and between each question and its alternatives occurs in a simple way, the design uses only the 'tab' or 'down arrow' tabs for the forward function and 'up arrow' and 'shift-tab' for the return function, allowing navigation between the question and the alternatives, advancing or returning as the participant chooses.

In order to allow the application to be used autonomously by the person with total visual impairment, Accessible ENEM announces, during navigation, the place where the user is in the application (module, function, text and others), and the action that he/she can perform (choose the alternative, record the piece of writing or navigate to an unanswered question). In addition, it informs, after the interaction, which action was performed and asks for confirmation. The following describes the features of the application.

3.4.1 Module 1 – Identification

The identification page allows the participant to register his/her name and ID number.

3.4.2 Module 2 - Questions

The multiple choice question module considers the different formats of questions and alternatives identified during the requirement survey, namely: test question and alternatives in text, statement and question with text and image.

In this module the candidates can listen to the test question, the description of the images and the alternatives, as well as read and re-read the texts as many times as necessary according to their need and command. After choosing the alternative, the application announces the message requesting confirmation of the chosen alternative and automatically navigates to the next question. Figure 1 shows the question module page.

Figure 1
Accessible ENEM question module¹²

The question module has additional tabs to help the user perform the test that allow him/her to: navigate between the application modules, navigate between the exam subjects, access the list of unanswered questions and access a specific question by its number.

3.4.3 Module 3 – Writing test

In this module, the candidate can listen to the writing test proposal, the supporting texts, the descriptions of the images related to the theme and the instructions for writing. In addition, he/she can type and proofread his/her piece of writing, reading and re-reading the text with autonomy. The application controls the length of the piece of writing, informing the participant how many characters are typed. After finishing it, the participant registers his/her piece of writing with the 'Save Copy' option.

3.4.4 Module 4 – Finishing the test

When the candidate completes the exam, the application checks for unanswered questions and whether the minimum and maximum numbers of character of the piece of writing are within the limits established for the exam. If inconsistencies are identified, the candidate hears the alert message advising that the piece of writing needs to be verified and the questions finalized and, then, he/she can correct the inconsistencies.

In the next item, we analyze the solutions implemented and the results observed during the evaluation performed by three volunteers with total visual impairment.

4 Discussion

Based on the analysis of the usability test results of the Accessible ENEM and the evaluation of satisfaction of the user when using the application, the main factors that demonstrate the autonomy and satisfaction (comfort, confidence, utility and pleasure) in the use of the application on the part of the people with disability were set. These participants collaborated to eliminate the difficulties encountered in the examination, mainly the ones related to the existing communication barriers. These factors are presented below, grouped into three sections that discuss, respectively, how the Accessible ENEM application favors access to the content of the exam, how the application facilitates the understanding of the content of the exam and how the Accessible ENEM allows the participant to demonstrate his/her knowledge with autonomy.

4.1 Accessible ENEM favors access to the exam content

Access to information on the content of the test is a basic condition to carry it out. In section 3.3.1, difficulties were reported around access to this information, due to the training of the reader to read the exam, as well as the difficulty in the complete reading of the extensive test in Braille, which has an average of 90 sheets per edition. The Accessible ENEM minimized these difficulties, favoring access as described below.

In relation to the difficulty of understanding symbols, such as in Mathematics and Chemistry, the application allows the substitution of ad hoc reading of content previously elaborated by professionals of specific areas of knowledge, in the form of a description text. In this way, the quality of the information and the independence of the education of the reader are guaranteed, since, during the reading of the question, the prepared texts are read automatically by the screen reader as if they were descriptions of images.

Regarding the reading of texts in Portuguese and other languages, Accessible ENEM is set up to read texts accurately, including English and Spanish texts, without accent, and automatically change the voice according to the language. For this, besides the accessible application, it is necessary to correctly parameterize the screen reader and use a synthesizer with adequate voice. Considering the exhaustion generated by handling the large print volume in Braille, the application can make the exam easier for readers who have less proficiency in reading the code.

The success of the Accessible ENEM in relation to favor the access to content depends, however, on some factors that are part of the context of use. First, content prepared by the examiners should be of good quality, accessible, follow the recommendations of digital accessibility (WCAG2, 2008) and be validated by means of screen reader testing. During application usability testing, we were able to verify how small content accessibility issues can compromise understanding.

4.2 Accessible ENEM facilitates the understanding of the content

In section 3.3.2, barriers related to the understanding of the exam content were reported, such as: lack of autonomy for reading; discomfort and embarrassment in the request to re-read the text; difficulty in understanding images; additional effort to understand the (spoken) reading, with the help of the reader.

According to the testimonies of the participants, the usual way of reading that people with total visual impairment have is the screen reader, which is used several hours a day. The Accessible ENEM allows the person with disability to read and re-read the text as many times as necessary, in a natural and effortless way to complete the task, allowing all the attention to be focused on the understanding of the text.

Also reported was the feeling of comfort in performing the test by listening to the screen reader, which maintains consistency in the voice from start to finish, without changing the tone or even speed during the reading or manifesting fatigue at the end of the test.

The success of the Accessible ENEM in favor of understanding the content depends on factors that are part of the context of use. For example, while the writing of mathematical formulas is standardized, their reading is not. There is scientific work on the verbalization of mathematical symbols, but a standardization for the Accessible ENEM application does not solve the lack of standardization in the teaching of mathematics in schools (Fuentes & Ferres, 2012).

Similarly, offering access to the text in a foreign language can be a barrier to the student who reads, but does not speak the foreign language. The participant could be given the option of listening to the texts in another language, with the voice in Portuguese (with an accent), as this is the way some people with disabilities are accustomed to listening to texts in another language. Oral comprehension in a foreign language is not required by the exam and is imposed on the person with total visual impairment, both with the application and with the reader.

As for the difficulty in understanding the images, the use of the application is not enough to completely solve this issue. The Accessible ENEM allows access to the descriptions of images, but, according to reports, access to the description of images with autonomy does not guarantee understanding. Participants said they prefer human assistance, which, despite spending time, can see and give examples of the images and clarify doubts about the more complex ones. However, this means favoring some participants and undermining others depending on the attitude and ability of the reader to describe the images.

There are aspects of accessibility that a computational application cannot provide in the context of an exam such as ENEM. They depend on pedagogical aspects and the public policies of the exam. Even if there is a description of images, it seems appropriate to evaluate the possibility of elaborating a differentiated test with questions that evaluate the knowledge without using visual means (images) for the candidates who do not see. The concept of disability based on the principle of equality implies recognizing difference in order to offer conditions of equality (Capelline, & Rodrigues, 2012). Therefore, we understand that in order to solve this issue, it is appropriate to use positive discrimination, that is, the creation of a differentiated test would be appropriate to achieve equality of conditions.

4.3 Accessible ENEM allows the participant to demonstrate knowledge with autonomy

As mentioned in section 3.3.3, the lack of autonomy to write the essay, as well as the embarrassment caused by the proofreading process - which requires reading and re-reading the text - and the lack of confidence in the transcription of the response card and writing test are barriers to express the knowledge of the candidate with disability. Thus, the Accessible ENEM presents the solutions that follow.

With the use of the Accessible ENEM, the participant has the autonomy to write the essay quietly, which facilitates the elaboration process, writing and proofreading the text. He/she may perform this activity without the discomfort and embarrassment of needing the assistance of another person accompanying each part of the process.

The use of Accessible ENEM improves the confidence of the person with total visual impairment in the writing result, which, with the application, only depends on the participant him/herself, eliminating the possibility of spelling mistakes that may occur with human assistance during transcription. Moreover, according to reports, the confidence of the participant improves, which helps to accomplish the task with freedom and autonomy, that is, without external factors that can disrupt creation.

The Accessible ENEM allows the person with disability to register his/her own answers, have control of the questions that have not been answered and control the size (minimum and maximum) allowed for the piece of writing. These factors increase the confidence of the person with total visual impairment and the possibility that their result is more accurate, that is, the actual performance of the person with disability in the test, since it is not subject to possible failure committed by the reader or subject to a disagreement in the communication with the reader. As one participant said during the study, a 'B' pronounced by the reader can easily be mistaken for a 'D'.

4.4 Difficulties observed in the use of accessible ENEM

Still in relation to the context of use, the tests showed that some of the Accessible ENEM resources for the candidate to control the test, such as the 'Mudar Matéria' (Change subject) and 'Questões Sem Resposta da Matéria' (Unanswered questions of the subject), were neither used or identified by the participants. It was found that it is necessary for the person with total visual impairment to have access to the application prior to the examination for training.

The difficulty of the test participants in adapting to different keyboards than their own was observed. In the project, after detecting the problem related to the keyboard, three types of keyboard to choose from were offered to the person with disability before the test. In the same way, INEP may include the keyboard as an accessibility resource, to be chosen by the person with total visual impairment at the time of registration, since INEP '[...] contacts each requester to confirm the requested resource' (Ministério da Educação, 2012a).

5 Conclusion

This study indicates the use of an application with digital accessibility as a way to make the ENEM test available for people with total visual impairment, and because there is a demand for adaptations, the application meets this demand and it is a right of the person with disability, as stipulated by law.

The study showed that there are barriers in the current process that limit access to information and make the comprehension of the content of the test difficult, besides this it hinders the expression of knowledge by the person with total visual impairment. From the tests and testimonials of participants with total visual impairment, we concluded that the application facilitates access to the information of the test content, eliminates the additional effort to understand the texts (spoken) by the reader and allows to perform the writing test and control its proofreading.

With our study, we concluded that the Accessible ENEM application allows the person with total visual impairment to perform the test with autonomy, concentrating better on the evaluation, without being affected by external factors. The use of the application provides equality of conditions for the person with total visual impairment during the examination. However, the context in which this study is inserted is complex and interdisciplinary. The effort to implement a nationwide exam, such as ENEM, in a fully accessible form for the person with total visual impairment, is certainly greater than the development of an accessible application. It necessitates the involvement of numerous other members of the process, in addition to the person with total visual impairment, as was done in this research. Special education professionals, those responsible for developing accessible content, information security specialists, those responsible for the application distribution logistics and computer equipment, social and educational inclusion policy managers, readers, and various of other stakeholders should be consulted to promote changes in the process, and in the careful selection of technology and training of people.

An example of this complexity is that the use of technology is itself a potential barrier: students who have not had high exposure to the computer or the use of the screen reader may feel intimidated to take the test with a resource they are unfamiliar with. For this, training mechanisms would have to be made available and disseminated, such as simulated exams using the application through the internet. In this sense, it is prudent that the application is offered as another alternative resource, along with other resources currently available (test in Braille and human assistance) in order to be an option to the person with total visual impairment as stated in the Brazilian Law of Inclusion (Decreto n º 13.146, 2015).

Considering these aspects, the study fulfills the purpose of showing the viability of the technology, but it is only part of the solution to obtain accessibility in university selective processes. It is evident the need for new research and investment in projects and actions with the aim of developing a technological solution according to the concept of universal design that can be used by anyone regardless of the disability and allowing for different types of exams, consisting of a digital platform with real conditions of equality in the selective processes.

• 2
  Financial Support: Coordination of Improvement of Higher Education People (CAPES) - 24 months of master's degree and National Council of Scientific and Technological Development (CNPq) - 12 months of Scientific Initiation scholarship.
• 7
  Reading support: 'Specialized reading service for people with visual impairment, intellectual disability, autism, attention deficit or dyslexia' (Ministério da Educação, 2012a). In ENEM, the readers act in pairs to read the test.
• 8
  Transcription support: 'Specialized service to fill out the multiple choice and piece of writing questions for participants who are unable to write or fill out the Response Card' (Ministério da Educação, 2012a).
• 9
  User Centered Design: '[...] approach to interactive systems development that aims to make systems usable and useful by focusing on the users, their needs and requirements [...]. This approach enhances effectiveness and efficiency, improves human well-being, user satisfaction, accessibility and sustainability; and counteracts possible adverse effects of use on human health, safety and performance' (International Organization for Standardization, 2010).
• 10
  Test of the reader: Test made with ink, which has the descriptions of the images.
• 11
  Universal design: conception of products, environments, programs and services to be used, to the greatest extent possible, by all people, without adaptation or specific design (Web Accessibility Initiative [WAI], 2010).
• 12
  Note of translation: The image shows an example of a question and one of the alternatives that the user can choose from. The menu below shows the following features: unanswered test questions of the subject, change subject, search, writing test, finish, help.

Referências

• ACT. Preparing for the ACT Test Special Testing 2016. Disponível em: <http://www.act.org/content/dam/act/unsecured/documents/preparing_for_ACTSplTst_Web.pdf>. Acesso em: 15 abr. 2017.
  » http://www.act.org/content/dam/act/unsecured/documents/preparing_for_ACTSplTst_Web.pdf
• BARBOSA, S. D. J.; SILVA, B. S. Interação humano-computador 2. ed. Rio de Janeiro: Leanpub, 2017.
• BERSCH, R. de C. R. Introdução à tecnologia assistiva 2017. Disponível em: <http://www.assistiva.com.br/Introducao_Tecnologia_Assistiva.pdf>. Acesso em: 10 abr. 2017.
  » http://www.assistiva.com.br/Introducao_Tecnologia_Assistiva.pdf
• BRASIL. Constituição da República Federativa do Brasil de 1988 Presidência da República. Casa Civil. Subchefia de assuntos jurídicos, Brasília, DF, 5 out. 1988. Tit. I, 1988. Disponível em: <http://www.planalto.gov.br/ccivil_03/Constituicao/Constituicao.htm>. Acesso em: 14 abr. 2017.
  » http://www.planalto.gov.br/ccivil_03/Constituicao/Constituicao.htm
• BRASIL. Lei n º 9.394, de 20 de dezembro de 1996 Estabelece as diretrizes e bases da educação nacional. Presidência da República. Casa Civil. Subchefia de assuntos jurídicos. Brasília, DF, 1996. Disponível em: <http://www.planalto.gov.br/ccivil_03/Leis/L9394.htm>. Acesso em: 14 abr. 2017.
  » http://www.planalto.gov.br/ccivil_03/Leis/L9394.htm
• BRASIL. Decreto n° 3.298, de 20 de dezembro de 1999. Regulamenta a Lei no 7.853, de 24 de outubro de 1989, dispõe sobre a Política Nacional para a Integração da Pessoa Portadora de Deficiência, consolida as normas de proteção, e dá outras providências. Disponível em: <http://www.planalto.gov.br/ccivil_03/decreto/d3298.htm>. Acesso em: 10 maio. 2017.
  » http://www.planalto.gov.br/ccivil_03/decreto/d3298.htm
• BRASIL. Decreto n º 5.296, de 2 de dezembro de 2004. Regulamenta as leis 10.048 e 10.098 de 2000. Disponível em: <http://www.planalto.gov.br/ccivil_03/_ato2004-2006/2004/decreto/d5296.htm>. Acesso em: 14 abr. 2017.
  » http://www.planalto.gov.br/ccivil_03/_ato2004-2006/2004/decreto/d5296.htm
• BRASIL. Secretaria Especial dos Direitos Humanos. Subsecretaria Nacional de Promoção dos Direitos da Pessoa com Deficiência. Comitê de Ajudas Técnicas. Tecnologia Assistiva Brasília, DF: CORDE, 2009a.
• BRASIL. Decreto n º 6.949, de 25 de agosto de 2009. Promulga a Convenção Internacional sobre os Direitos das Pessoas com Deficiência e seu Protocolo Facultativo, assinados em Nova York, em 30 de março de 2007. 2009b. Disponível em: <http://www.planalto.gov.br/ccivil_03/_ato2007-2010/2009/decreto/d6949.htm>. Acesso em: 10 abr. 2017.
  » http://www.planalto.gov.br/ccivil_03/_ato2007-2010/2009/decreto/d6949.htm
• BRASIL. Ministério da Educação. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. Diretoria de Avaliação da Educação Básica. O atendimento diferenciado no ENEM, 2012a. Disponível em: <http://download.inep.gov.br/educacao_basica/enem/nota_tecnica/2012/atendimento_diferenciado_enem_2012.pdf>. Acesso em: 13 abr. 2017.
  » http://download.inep.gov.br/educacao_basica/enem/nota_tecnica/2012/atendimento_diferenciado_enem_2012.pdf
• BRASIL. Ministério da Educação. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. Microdados - INEP 2012b. Disponível em: <http://portal.inep.gov.br/microdados>. Acesso em: 13 abr. 2017.
  » http://portal.inep.gov.br/microdados
• BRASIL. Decreto n º 13.146, de 6 de julho de 2015. Institui a Lei Brasileira de Inclusão da Pessoa com Deficiência (Estatuto da Pessoa com Deficiência). 2015a. Disponível em: <http://www.planalto.gov.br/ccivil_03/_Ato2015-2018/2015/Lei/L13146.htm>. Acesso em: 10 abr. 2017.
  » http://www.planalto.gov.br/ccivil_03/_Ato2015-2018/2015/Lei/L13146.htm
• BRASIL. Ministério da Educação. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. Provas e Gabaritos - ENEM 2015b. Disponível em: <http://portal.inep.gov.br/provas-e-gabaritos>. Acesso em: 13 abr. 2017.
  » http://portal.inep.gov.br/provas-e-gabaritos
• BRASIL. Ministério da Educação. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. Balanço da Inscrição: Enem 2016, 2016a. Disponível em: <http://portal.mec.gov.br/index.php?option=com_docman&view=download&alias=41461-balanco-inscricao-enem-2016-pdf&Itemid=30192>. Acesso em: 11 abr. 2017.
  » http://portal.mec.gov.br/index.php?option=com_docman&view=download&alias=41461-balanco-inscricao-enem-2016-pdf&Itemid=30192
• BRASIL. Edital nº 10, de 14 de abril de 2016. O presidente do Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep) torna pública a realização da edição do Enem 2016. Ministério da Educação. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, Brasília, DF, 2016b. Disponível em: <http://download.inep.gov.br/educacao_basica/enem/edital/2016/edital_enem_2016.pdf>. Acesso em: 13 abr. 2017.
  » http://download.inep.gov.br/educacao_basica/enem/edital/2016/edital_enem_2016.pdf
• BRASIL. Ministério da Educação. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. SISU - Sistema de Seleção Unificada, 2017a. Disponível em: <http://sisu.mec.gov.br/sisu>. Acesso em: 13 abr. 2017.
  » http://sisu.mec.gov.br/sisu
• BRASIL. Ministério da Educação. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. SISUTEC, 2017b. Disponível em: <http://portal.mec.gov.br/index.php?option=com_content&view=article&id=34811&Itemid=30451>. Acesso em: 13 abr. 2017.
  » http://portal.mec.gov.br/index.php?option=com_content&view=article&id=34811&Itemid=30451
• CAPELLINE, V. L. M. F.; RODRIGUES, O. M. P. R. Educação inclusiva: fundamentos históricos, conceituais e legais. Bauru: UNESP/FC, 2012. (Coleção Práticas educacionais inclusivas, 2).
• CAPES. Comissão de Aperfeiçoamento de Pessoal do Nível Superior. Tabela de áreas do conhecimento / Avaliação 2014. Disponível em: <http://www.capes.gov.br/avaliacao/instrumentos-de-apoio/tabela-de-areas-do-conhecimento-avaliacao>. Acesso em: 13 maio. 2017.
  » http://www.capes.gov.br/avaliacao/instrumentos-de-apoio/tabela-de-areas-do-conhecimento-avaliacao
• COLLEGEBOARD. New 2017 Services for students with disabilities policies - The College Board Disponível em: <https://www.collegeboard.org/students-with-disabilities/whats-new>. Acesso em: 15 abr. 2017.
  » https://www.collegeboard.org/students-with-disabilities/whats-new
• DIMITROVA-RADOJICHIKJ, D. B. Students with visual impairments: braille reading rate. International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), Serbia, v. 3, n. 1, p. 5, 2015.
• DUARTE, E. R. et al. Estudo de caso sobre a inclusão de alunos com deficiência no Ensino Superior. Revista Brasileira de Educação Especial, Marília, v. 19, n. 2, p. 289-300, jun. 2013.
• FUENTES S. J.; FERRES L. Improving accessibility to mathematical formulas: the Wikipedia Math Accessor. New Review of Hypermedia and Multimedia, v. 18, n. 3, p. 183-204, set. 2012.
• FUJIYOSHI, M.; FUJIYOSHI, A.; AOMATSU, T. New testing method for the dyslexic and the newly blind with a digital audio player and document structure diagrams. In: MIESENBERGER, K. et al. (Orgs.). Computers Helping People with Special Needs Heidelberg: Springer, Berlin, 2010. p. 116-123.
• GALVÃO, T. A. F. Tecnologia assistiva para uma escola inclusiva: apropriação, demanda e perspectivas. 2009. 346 f. Tese (Doutorado em Educação) - Faculdade de Educação, Universidade Federal da Bahia, Salvador, 2009.
• INTERNATIONAL ORGANIZATION FOR STANDARDIZATION. ISO 9241-210: ergonomics of human-system interaction - Part 210: human-centred design for interactive systems. 2010. Disponível em: <https://www.iso.org/obp/ui/#iso:std:iso:9241:-210:ed-1:v1:en>. Acesso em: 5 abr. 2017.
  » https://www.iso.org/obp/ui/#iso:std:iso:9241:-210:ed-1:v1:en
• INTERNATIONAL ORGANIZATION FOR STANDARDIZATION. International standard - ISO/IEC- 25010: system and software engineering - System and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models. Geneva: ISO, 2011.
• LERIA, L. A. Acessibilidade digital em processos seletivos universitários para pessoa com deficiência visual: desenvolvimento do aplicativo ENEM Acessível. 2016. 187 f. Dissertação (Mestrado em Engenharia da Informação), Universidade Federal do ABC, Santo André, 2016.
• LERIA, L. A.; FERREIRA, L. A.; FRAGA, F. J. ENEM Acessível: desenvolvimento de aplicativo para participação autônoma da pessoa com deficiência visual total. In: SEMINÁRIO NACIONAL DE INCLUSÃO DIGITAL, 4., 2016, Passo Fundo. Anais... Passo Fundo: Anais do SENID, 2016. Disponível em: <http://senid.upf.br/2016/images/pdf/152021.pdf>. Acesso em: 13 fev. 2018.
  » http://senid.upf.br/2016/images/pdf/152021.pdf
• MARTINS, D. A.; LEITE, L. P.; LACERDA, C. B. F. de. Políticas públicas para acesso de pessoas com deficiência ao ensino superior brasileiro: uma análise de indicadores educacionais. Ensaio: Avaliação e Políticas Públicas em Educação, Rio de Janeiro v. 23, n. 89, p. 984-1014, dez. 2015. Disponível em: < http://www.scielo.br/pdf/ensaio/v23n89/1809-4465-ensaio-23-89-0984.pdf>. Acesso em: 22 jul. 2017.
  » http://www.scielo.br/pdf/ensaio/v23n89/1809-4465-ensaio-23-89-0984.pdf
• NV ACCESS. NonVisual Desktop Access - NVDA. 2015. Disponível em: <https://www.nvaccess.org/>. Acesso em: 17 abr. 2017.
  » https://www.nvaccess.org/
• SANCHEZ, J.; ESPINOZA, M.; CAMPOS, B. Design and usability evaluation of an audio-based college entrance exam for students with visual disabilities. Virtual Reality & Associated Technologies, Gothenburg, p. 2-4, 2014. Disponível em:< http://www.icdvrat.org/2014/papers/ICDVRAT2014_S10N4_Sanchez_etal.pdf>. Acesso em: 22 jul.2017.
  » http://www.icdvrat.org/2014/papers/ICDVRAT2014_S10N4_Sanchez_etal.pdf
• SANTOS, J. M. C. T. Exame Nacional do Ensino Médio: entre a regulação da qualidade do Ensino Médio e o vestibular. Educar em Revista, Curitiba, v. 0, n. 40, p. 195-205, 2011.
• SILVA, M. C. V.; MELETTI, S. M. F. Estudantes com necessidades educacionais especiais nas avaliações em larga escala: prova Brasil e ENEM. Revista Brasileira de Educação Especial, Marília, v. 20, n. 1, p. 53-68, 2014.
• TORRES, E. F.; MAZZONI, A. A.; ALVES, J. B. da M. A. A acessibilidade à informação no espaço digital. Ciência da Informação, Brasília, v. 31, p. 83-91, 2002.
• W3C-BRASIL. Cartilha de Acessibilidade na Web - W3C Brasil. 2013. Disponível em: <http://www.w3c.br/pub/Materiais/PublicacoesW3C/cartilha-w3cbr-acessibilidade-web-fasciculo-I.html>. Acesso em: 5 abr. 2017.
  » http://www.w3c.br/pub/Materiais/PublicacoesW3C/cartilha-w3cbr-acessibilidade-web-fasciculo-I.html
• WEB ACESSIBILITY INITIATIVE. Accessibility, usability, and inclusion. 2010. Disponível em: <https://www.w3.org/WAI/intro/usable#introduction>. Acesso em: 5 abr. 2017.
  » https://www.w3.org/WAI/intro/usable#introduction
• WAZLAWICK, R. S. Metodologia de Pesquisa para Ciência da Computação 2. ed. Rio de Janeiro: Elsevier, 2008.
• WCAG2. Web Content Accessibility Guidelines (WCAG) 2.0. 2008. Disponível em: <https://www.w3.org/TR/WCAG20/>. Acesso em: 5 abr. 2017.
  » https://www.w3.org/TR/WCAG20/

Publication Dates

• Publication in this collection
  Jan-Mar 2018

History

• Received
  23 July 2017
• Reviewed
  06 Mar 2018
• Accepted
  12 Mar 2018