MACHINE TRANSLATION: TEACHING AND LEARNING ISSUES

TRADUÇÃO AUTOMÁTICA: QUESTÕES DE ENSINO E APRENDIZAGEM

ABSTRACT

Considering the boost in technological development, and that machine translation has been widely used by both industry and academia, the main goal of this paper is to describe possibilities for the incorporation of a basic machine translation module in a Translation program at university level. The didactic proposal herein presented aims at providing Translation students ways of identifying general output results different modern machine translation systems have in common. Through a task-based teaching approach, students may assess machine translation strengths and weaknesses in order to make better use of this technology.

Keywords:
machine translation; translation technologies; translation teaching and learning

RESUMO

Considerando o impulso no desenvolvimento tecnológico e o fato de que a tradução automática tem sido amplamente utilizada tanto pela indústria como pelo meio acadêmico, este artigo descreve possibilidades de incorporação de um módulo de tradução automática em cursos de graduação em Tradução. A proposição didática que ora se apresenta busca fornecer aos estudantes de Tradução formas de identificação dos resultados gerais comumente exibidos por diferentes sistemas de tradução automática da atualidade. Por meio de uma abordagem de ensino baseada em tarefas, os alunos podem estimar as potencialidades e limitações da tradução automática, visando à utilização mais eficaz desta tecnologia.

Palavras-chave:
tradução automática; tecnologias de tradução; ensino e aprendizagem de tradução

1. CONTEXTUALIZING MACHINE TRANSLATION

For a long time machine translation (MT) has been rejected in the professional and educational environments, particularly due to its alleged attempts in trying to replace human translators. Defined as a fully automatic process that starts with a text in one language and produces a corresponding text in another language (MELBY, 2020), MT outputs were traditionally thought to be used as is, i.e., the way it was performed by a machine.

The long history of MT dates back to the 1950s in the United States, when it was believed that this technology could be successfully operated by a rule-based approach, by which researchers tried to program computers to process natural language using grammar rules and linguistic patterns. Since then, researchers have been trying to achieve an independence of the systems from human intervention, searching for developing a fully automatic approach that could be used in different situations with acceptable quality.

The dissemination amongst translators of the potential independence of MT systems, and the fear they have of losing their place in the working market, have been causing these professionals to oppose to any sort of interaction with machine translation technologies, including post-editing approaches, which today are in great demand in translation agencies. As Gaspari (2001, p. 35) explains, there has always been tension or even conflict between translator´s traditional approach, as they strive for delivering high-quality translations no matter the translational situation, and the possible alternatives offered by different degrees of technological intervention. Doherty et al. (2018) agree that:

It is also true that for decades there was hardly any exchange between MT researchers and developers on the one hand, and professional translators and translation theorists on the other; this was mostly because translators have historically tended to see MT as a threat […], and (like translation theorists) the difficulties that MT faced in the days of rule-based systems were too banal from their point of view to take MT seriously […]. (DOHERTY et al., 2018, p. 99)

In the early 1960s, when the results of tests implemented with the available machine translation systems were released, they showed that machine translation outputs were insufficient to deal with the complexity of natural language problems. An important report produced in 1966 by the Automatic Language Processing Advisory Committee (ALPAC) put an end to the research on machine translation in the US, demonstrating that the high cost of its implementation did not offset the tiny results achieved so far (HUTCHINS, 2015; WAY, 2018).

Arguably, this scenario has changed quite considerably, and research in MT has persisted in other countries, such as Canada, France, and Germany. For the sake of illustration, in Montreal (Quebec, Canada), research on MT began in 1970 using syntactic transfer techniques for English−French translations. One of the achievements of the TAUM project (Traduction Automatique de l’Université de Montréal - Machine Translation Project created by University of Montreal) was METEO system for translating weather forecasts. Designed specifically for the restricted vocabulary and limited syntax of meteorological reports, METEO has been successfully operating since 1976 (MARTINS; NUNES, 2005; BOWKER, 2020).

As explained by Shiwen and Xiaojing (2015), the significance of linguistic knowledge in MT has been repeatedly reflected upon, which leads to an ever-growing understanding of the role that linguistic knowledge plays in MT. They state that:

A more fundamental issue, however, is how to represent the linguistic knowledge so that it can be processed and utilized by MT systems. Basically, there are two types of formalized knowledge representations: dictionaries and grammar rules on the one hand, and corpora on the other. As explicit representations, the former adopt formal structures, such as relational databases and rewrite rules; as implicit representations, the latter use linear strings of words. (SHIWEN; XIAOJING, 2015, p. 195)

Thanks to computing power and machine learning applied to huge bitext (parallel corpora), besides the aforementioned rule-based approach (followed by direct, transfer and interlingua models in time), new approaches and methods have been applied to MT systems. Broadly speaking, they can be named as:

• corpus-based statistical approach, where computers are trained with parallel corpora and use probabilities. According to Kenny and Doherty (2014), in a statistical approach “rather than trying to encode a priori in the form of dictionaries, grammars and knowledge bases, all the linguistic and world knowledge required to translate a text from one language into another (the approach taken in rule-based and knowledge-based MT) simply learn how to translate from already existing human translations. In practice, such learning involves the induction of statistical models of translation from parallel corpora, that is, source texts and their human translations” (p. 278); and more recently,

• neural networks, or, neural machine translation (NMT), where information processing system that is inspired by the way biological nervous systems, such as the brain, process information (BOWKER; BUITRAGO CIRO, 2019). The neural network machine translation approach finds patterns, such as contextual clues around the source phrase (TORAL; WAY, 2018).

Different approaches were created as relatively new methods and techniques for machine translation. Knowledge-based and example-based approaches are examples of such methods and techniques, and they can be considered as extensions of linguistic transfer rule-based approach and statistical approach.

At the present time, most machine translation technologies use a hybrid approach so that they can take advantage of the previous methods and techniques applied to date. It is worth mentioning that albeit emerged in the 1950s, rule-based machine translation methods perform well between very similar languages that could be considered dialects of each other (MELBY, 2020).

Considering the above, one can say that MT has evolved from different approaches, and neural machine translation, the last generation of MT systems, is rapidly becoming the dominant data-driven approach. Nevertheless, according to Melby (2020), sufficient training data for a viable NMT system are available only for a handful (perhaps twenty) of the over-four-thousand languages in the world. According to the author, for the rest (over 99%), either rule-based or statistical machine translation approaches, or, in most cases, human translation, are the only options.

Despite different possibilities of recounting the history of MT paradigms, which is beyond the scope of this paper, current trends show that MT research has been uniting linguists and computer engineers in long-lasting studies involving morphological, syntactic and semantic analyses, examples extracted from numerous pairs of source sentences and their respective translations. Besides that, some other research initiatives on MT use additional parallel data, in which the source text is synthetically created, i.e., the source text is machine translated from the target language to be used as a dataset, which has been reported to be a successful way of integrating target-language monolingual data into neural machine translation. Moreover, in-domain data training, for example, e-books are converted to plain texts through specific software, such as Calibre support tools (https://calibre-ebook.com/), and are then sentence-split for a certain pair of languages, tokenized and finally sentence-aligned, as explained by Toral and Way (2018).

The facts that MT systems are getting better because they are using all traditional and modern MT approaches, and the more they are online and freely accessible to a wide range of external users, the better they get (PYM, 2013), are again attracting the attention not only of researchers within Translation Studies, Computer Engineering and Computational Linguistics, but also of translation teachers and students alike, who are inevitably touched by the impact MT systems have on the translator’s work and education.

Concerning the educational scenario, the greatest challenges in teaching machine translation technology to translation students are precisely the demystification of the idea that MT systems available today have the alleged early goal of replacing the translator, and that machine translation requires lots of human interventions, which presumably make them ineffective.

In view of MT’s developments throughout history, and its ever-growing use in both industry (trade, localization of software apps, websites, audiovisual materials, video games, crowdsourcing projects) and academia (scholarly communication), shouldn’t Translation programs formally approach the topic?

Rather than replacing human translators, a MT system can be used as a standalone tool, or as a specific feature of many popular technologies, such as translation memory system (TMS), allowing the options offered by the system to be used for informational purposes only, or as a pre-translation of a particular source text, subsequently guiding the translator’s work to the adjustments and corrections phase, the so-called post-editing (ZETZSCHE, 2016; O’BRIEN, 2002). In Vieira’s words: “post-editing of MT has been carried out since the early days of MT technology” (VIEIRA, 2020, p. 319).

Accordingly, conceived as a didactic proposal originated from a qualitative class research of naturalistic bias (detailed below), the main goal of this paper is to describe possibilities for the incorporation of a basic machine translation module in a Translation program at university level. The purpose is to provide students ways of identifying general output results different modern machine translation systems have in common, as well as their strengths and weaknesses¹. As Doherty et al. (2018, p. 97-98) aptly affirm, through a sequence of activities involving machine translation teaching, students may realize the need for a basic machine translation literacy, in order to deal with (un)predictable communicative situations and (mis)understandings during its use, and decide how to benefit from MT outputs. Way explains that gaining awareness and training on machine translation,

[…] applies also to translators who complain that MT quality is too poor to be used in their workflows; in order to decide that with some certainty - rather than rejecting MT out-of-hand merely as a knee-jerk reaction to the onset of this new technology - the impact of MT on translators’ work needs to be measurable. (WAY, 2018, p. 160)

The requirement to keep translation students abreast of MT technological developments is arguably crucial because these future professionals may probably be asked to offer a diverse and already existing range of translation related services including, e.g., (a) organizing the language resources and terminological assets to be used via CAT tools in large multilingual translation and localization projects, (b) pre-editing, (c) post-editing, (d) diagnostic evaluation of MT systems (BOWKER; MARSHMAN, 2010; BOWKER; BUITRAGO CIRO, 2019), and (e) training of a MT system for free and/or open-source MT platforms, such as Apertium, that deals with divergent language pairs (such as Portuguese-Catalan or English-Galician or French-Occitan) (see https://www.apertium.org), to name but a few.

2. QUALITATIVE CLASS RESEARCH OF NATURALISTIC BIAS: A START-POINT FOR A DIDACTIC PROPOSAL

Most well-established translator training programs at university level now possibly include components focusing on the use of CAT tools and other translation technologies with their related skills, most notably MT and post-editing, a development which, in my view, is certainly positive and responsive to industry needs. However, Translation programs should not only prepare students for the working market, but also to acquire a critical view of theories and practices they are exposed to during their educational program. As claimed by Hurtado Albir (2008, p. 18), it is not just a question of training professionals, but of educating individuals, who in a near future will be able to learn how to learn, to be critical, to better communicate and cooperate amongst their peers. Given that, how can we prepare students to critically use MT technologies? How can we help translation students to critically perceive MT’s strengths, weaknesses and its possible usages remaining them abreast of latest technological changes?

One way to design a module on MT could get started firstly identifying what students know about MT, and how they assess its outputs. Are students familiar with the strengths and weaknesses of MT systems that are now commonplace in the industry, especially considering the media hyperbole on the potential of the neural approach adopted by Google Translate after 2016 (MOORKENS, 2018)?

Alluding to the methodological concept previously mentioned, this didactic proposal is the result of a qualitative class research of naturalistic bias², i.e., a previous systematic observation of how translation students have used and assessed MT outputs as it has occurred in class. Even though this paper is not a presentation of an exploratory research, it emerges from data collected in 2020 in a translation technology course I am responsible for in a Brazilian Translation program. The class had approximately 25 Brazilian Translation students of an undergraduate Translation program. The students typically range in age from around 20 to 60, with an average age of 25. After studying the main concepts on MT, its definitions and main approaches, students were asked to read a 100-word English abstract of a Civil Engineering research paper; machine translate it from English to Portuguese through free online machine translation systems they were supposed to choose from a given list; and, after that, they were asked to assess MT outputs in order to highlight their strengths and weaknesses.

Overall, students reported that the abstract had been quite sufficiently³ translated by MT systems, and three out of 25 reported that before publishing the abstract in Portuguese, or delivering it to a supposed client, they should check and control some terminological entries. Even though students have shown a tendency to accept literal translations, their twofold perceptions are not totally misguided. For Hutchins (2015), there are at least two main usages of MT outputs: one that uses MT systems to get some idea of the contents (the “basic message”) of foreign texts and is not concerned about the quality of translations, referred to by the author as assimilation purpose; the other use, in contrast, is referred as dissemination or publishing purpose, where quality matters.

Based on this previous systematic observation of the phenomena as it has occurred in class, this didactic proposal is designed to put in evidence some strengths and weaknesses 10 different MT systems show other than terminological issues students have previously detected, aggregating knowledge to their twofold perceptions.

Vieira (2020) and Yamada (2019) affirm that translation students usually have insufficient error detection rate when post-editing machine translated texts for dissemination or publishing purposes, whether applied to neural or statistical machine translation systems, even though neural machine translation, as the last generation of MT approach, is supposedly to present superior performance.

While MT approaches, text genres, how source language is written (and analyzed) and how the target translation output is generated all interfere in the final result, translation students can be prepared to identify common merits and drawbacks in the outputs of different MT systems of today.

3. DESIGNING A DIDACTIC PROPOSAL FOR MACHINE TRANSLATION TEACHING: TASK-BASED APPROACH, MATERIALS, AND SYSTEMS

Both in Education field (PIMENTA, 1995; SAVIANI, 2007; MELO, 2018) and Translation Studies (FIOLA, 2003; GONZÁLEZ-DAVIES, 2004; KIRALY, 2000; KIRALY et al., 2016; ESQUEDA, 2018; 2020b), teaching approaches can be understood as theories, or theoretical paradigms that support the adoption of a set of teaching methods and techniques. While an approach is a set of theoretical assumptions related to teaching and learning process (teaching philosophy), a method is a general pre-set plan for systematic presentation of certain content (didactic proposal / what to teach). Compatible with the teaching approach and method, and balancing teacher-student relationship in class, are the teaching techniques, conceived as specific activities to be accomplished in the classroom (pedagogy / how and why to teach).

Amongst some different existing translation teaching approaches, such as early and conventional linguistic approaches (MOUNIN, 1975; CATFORD, 1965), or more recent ones such as learning by objectives (DELISLE, 2013), functionalist (NORD, 2016), process-centered (GILE, 2009), cognitivist (ERICSSON, 2000), situational (or realistic) (RISKU, 2016), personal or autonomous (ROBINSON, 2002), socioconstructivist (KIRALY, 2000), or humanistic (ARROJO, 1993) approaches, and their related methods and techniques, to name but a few, this didactic proposal finds support in a competence-based approach (ESQUEDA, 2020a; HURTADO ALBIR, 2017). Related didactic and pedagogical methods and techniques are therefore embedded in task-based learning, whose class activities prepare students to deal with a representative task that belongs to situations of the real world, intentionally designed with a tangible objective, structure and sequence. Although Hurtado Albir (2008) advocates that competence-based approach may integrate task-based, problem-based, collaborative or project-based teaching methods and techniques, task-based learning seems to be appropriate to conceive a module to envisage basic technological issues on MT, once some initial knowledge and experience need to be progressively acquired by students, in an autonomous and critical pace, before they work in large translation projects involving peers and future clients (HURTADO ALBIR, 2015).

3.1 Task-based learning

As already mentioned, this paper describes a machine translation module that can be incorporated across a Translation program. Through an initial module, at the very beginning of the program, teachers can foster the necessary technical and linguistic skills that undergraduates will acquire to be successful in assessing and using MT.

Since Translation programs in Brazil, and around the world, will vary, this proposal, different from a specific module involving the technical and technological architecture of MT systems (KENNY; DOHERTY, 2014), is basic in nature, and ought to be adapted, whether to general or domain specific translation courses. Subsequently, cross-module integration of machine translation theory and practice in an intermediate or advanced-spectrums are advisable, as an attempt to foster networked learning (KELLY, 2005), and avoid curriculum compartmentalization.

With exercises that help Translation students to identify and practice specific features on MT, this task-based didactic proposal includes five tasks and a self-evaluation report, forming a sequence of activities with the purpose of providing knowledge on the following questions: What are MT systems possible usages? Can MT systems of today and their different approaches perform differently in a determined translation task? Do MT systems present the same set of strengths and weaknesses? As stated by Garcia (2010), through understanding MT’s strengths and weaknesses, students get to know how to use a MT software for translating a specific text genre when taking into account not only information objectives, but also dissemination and publishing purposes (HUTCHINS, 2015) (See section 2).

Therefore, the sequence of activities includes⁴:

1. Task 1 - Reading an 100-word English abstract of a scientific paper written by Xu et al. (2014) on Civil Engineering for trigging content awareness;

2. Task 2 -Choosing MT systems to machine translate Xu et al.’s (2014) abstract;

3. Task 3 -Analyzing MT systems outputs and identifying their strengths;

4. Task 4 -Analyzing MT systems outputs and identifying their weaknesses;

5. Task 5 -Reporting orally MT systems performance amongst peers;

6. Final Task -Building a self-evaluation report where students individually comment, in a written form with three to five pages, what they have learned during the task concerning MT systems possible usages, their different approaches and performance in relation to a determined translation task, and their strengths and weaknesses.

3.2 Materials

With the purpose of describing a module that involves machine translation teaching, it is assumed that a group of approximately 25 students has received the request to translate a scientific abstract from English to Portuguese of approximately 100 words. The abstract used as example was extracted from Xu et al. (2014), a paper published in the international journal Construction and Building Materials, belonging to Elsevier Dutch Publishing company. The paper has been chosen for the sake of illustration because its translation from English to Portuguese has been authentically requested by a professor of the Civil Engineering Department of the same university where Translation students belong to (see section 2), so chances are that these students will be requested to provide this sort of service in the future (KIRALY, 2000; GONZÁLEZ-DAVIES, 2017). Even though Fonseca and Alves (2016) report on different theoretical and empirical studies’ claims on the number of words a text should contain for research and didactic purposes, a 100-text seems to be favorable in length and time. Figure 1 shows details related to Xu et al.’s (2014) abstract.

Figure 1
General information on Xu et al.'s (2014) abstract

Title of the paper	Evaluation of pennanent deformation of asphalt mixtures using different laboratory performance tests
100-word English abstract	The laboratory tests were conducted to evaluate the effects of polyester fiber on pennanent deformation of asphalt mixtures. The results indicate the fibers improve the deformation resistance of mixtures. The confinements in the partial triaxial test have (PTT) significant effects on the permanent strain of mixtures. The varying confinement in the PTT better simulates the actual confinements in pavements. A stronger correlation of test results from the PTT and the test was found. It is concluded that the addition of polyester fibers improves the mechanical performance of mixtures, and the PTT method is areliable new test method.
Reference	XU, T. et al. Evaluation of pennanent deformation of asphalt mixtures using different Laboratory performance tests. Construction and Building Materials, n. 53,p. 561-567,2014. Retrieved from: https://bit.ly/39vf5ka. Latest access: April 1 2020.

3.3 MT systems

Students can use machine translation systems of different MT paradigms (rule-based MT, statistical MT, neural MT, and hybrid) which cut right across the spectrum of MT systems as they are used today in real translation workflows or that are analyzed in the current MT literature herein referenced. The main idea is to select relevant machine translation systems and make students aware of a large set of common strengths and weaknesses they reveal. Figure 2 displays 10 free/commercial online machine translation systems chosen for the proposal. Columns alphabetically present the title of the system, their origins, the applied approach to systems they publicize in their websites, and their access link, respectively. It is worth mentioning that, to my knowledge, English-(Brazilian)Portuguese MT for Civil Engineering domain is inexistent and therefore was not included in this proposal, which would probably reveal different results. Kenny and Doherty (2014, p. 284) highlight the importance of in-domain MT systems:

Figure 2
Ten free/commercial online machine translation systems

Svstem	Origin	Applied approach	Retrieved frona
1 Babylon	Israel	Rule-based and hybrid5	https://translation.babylon-software.com/english/to-portuguese/
2 Bing - Microsoft	United States	Statistical and neural	https://www.bing.com/translator
3 Deepl	Germany	Neural	https://www.deepl.com/translator
4 Google Translate	United States	Neural	https://translate.google.com/
5 My tnemcry	Italy	Neural	https://mymemory.translated.net
6 Promt	Russia	Statistical, neural and hybrid	https://www.online-translator.com/?prmtlang=en
7 Systran	France	Rule-based, statistical and neural	https://www.systransoft.com/lp/machine-translation/
8 Tradukka	Ecuador	Rube-based	https://tradukka.com/translate
9 Wordlingo	United States	Rule-based, statistical, and hybrid	https://www.worldlingo.com/br/
10 Yandex	Russia	Statistical and neural	https://translate.yandex.com/

[…] translation and language models reflect the data on which they were trained. A system trained on legal texts, for example, will be more useful for translating legal texts than it will be for texts from other domains. Translators and translation companies who specialise in particular areas are the very people who are likely to possess the in-domain texts on the basis of which useful translation models can be trained for their own needs. Such specialised data constitute a valuable commodity, especially when their quality can be vouched for. (KENNY; DOHERTY, 2014, p. 284)

Hedberg (2019) reports favorably on Spanish-Swedish neural machine translation for the Civil Engineering domain she has trained for her thesis in comparison to Google Translate, but unfortunately the linguistic pair does not meet the objectives of this didactic proposal for Brazilian students.

Figure 2 shows that machine translation providers publicize that their systems use a determined approach (third column). Even though different approaches may employ different and hybrid techniques, using different computational algorithms, this didactic proposal intents to provide students basic knowledge on common strengths and weaknesses amongst systems, fostering the recognition and prediction of their performance for future tasks, regardless the approach they use. In doing so, translation teachers can address both linguistic gaps and machine translation potentialities and limitations students tend to overlook when it comes especially to dissemination or publishing purposes.

4. TEACHING MT THROUGH A TASK-BASED LEARNING

4.1 Task 1-Reading an 100-word English abstract of a scientific paper written by Xu et al. (2014) on Civil Engineering for trigging content awareness

Without any kind of pre-editing, students are supposed to read the 100-word abstract of Xu et al.’s (2014) scientific paper. Task 1 implies that, without going deeply into text genre, students may realize that scientific papers, especially the ones published in mainstream scientific journals, are considered one of the most prestigious sorts of academic publication, once they reveal the evolution of science in terms of how content is expressed amongst scholars. They usually demonstrate their own language and structure, divided into parts, such as Abstract, Introduction, Theoretical Issues, Methodology, Research Results and Analysis, and Final conclusions (OLOHAN, 2016). To be more effective in time, in this task students may voice out to peers general characteristics they recognize in scientific papers in general, and in Xu et al.’s (2014) abstract structure and content.

4.2 Task 2 -Choosing MT systems to machine translate Xu et al.’s (2014) abstract

Individually, in pairs, or groups of three, depending on the capacity of technology laboratories available at universities (or depending on remote or distance learning proposals), students will machine translate Xu et al.’s (2014) abstract in Task 2. The purpose is to use different MT systems in order to identify possible commonalities in their outputs. Once universities may encounter some financial constraints to buy paid MT systems and their APIs, Task 2 can be accomplished with free/commercial online MT systems, such as the ones listed in Figure 2. Although some of them commercialize their APIs, they all offer free machine translation engines in their websites.

4.3 Task 3 -Analyzing MT systems outputs and their strengths

Task 3 presupposes that students will analyze the strengths of MT systems. Figure 3 shows Xu et al.’s (2014) abstract machine translated by 10 different MT systems students are supposed to use.

Figure 3
Machine Trans ation outputs of Xu et al.'s (2014) abstract by the 10 different MT systems

System	Machine Translation Output
1 Babylon	Os testes de laboratório foram conduzidos para avaliar os efeitos da fibra de poliéster em deformação permanente de misturas de asfalto. Os resultados indicam as fibras melhoram a resistência a deformação de misturas. O confinements no ensaio triaxial parcial (PTT) têm efeitos significativos sobre a deformação permanente de misturas. O confinamento variando no melhor PTT simula o real confinements em pavimentos. Uma correlação mais forte dos resultados do teste do PTT e roda o teste de rastreio foi encontrada. Conclui-se que a adição de fibras de poliéster melhora o desempenho mecânico de misturas, e o método de PTT é um novo método de teste.
2 Bing Microsoft	Os testes laboratoriais foram realizados para avaliar os efeitos da fibra de poliéster na deformação permanente das misturas asfálticas. Os resultados indicam que as fibras melhoram a resistência à deformação das misturas. Os confinamentos no teste triaxial parcial (PTT) têm efeitos significativos sobre a tensão permanente das misturas. O confinamento variado no PTT simula melhor os confinamentos reais nas calçadas. Uma correlação mais forte dos resultados dos testes do PTT e do teste de rastreamento das rodas foi encontrada. Conclui-se que a adição de fibras de poliéster melhora o desempenho mecânico das misturas, e o método PTT é um novo método de teste confiável.
3 DeepL	Os testes de laboratório foram realizados para avaliar os efeitos da fibra de poliéster na deformação permanente das misturas asfálticas. Os resultados indicam que as fibras melhoram a resistência à deformação das misturas. Os confinamentos no teste triaxial parcial (PTT) têm efeitos significativos sobre a deformação permanente das misturas. O confinamento variável no PTT simula melhor os confinamentos reais em pavimentos. Foi encontrada uma correlação mais forte entre os resultados dos testes do PTT e do teste de rastreamento de rodas. Conclui-se que a adição de fibras de poliéster melhora o desempenho mecânico das misturas, e o método PTT é um novo método de teste confiável.
4 Google Translate	Os testes laboratoriais foram realizados para avaliar os efeitos da fibra de poliéster na deformação permanente de misturas de asfalto. Os resultados indicam que as fibras melhoram a resistência à deformação das misturas. Os confinamentos no teste triaxial parcial (PTT) têm efeitos significativos sobre a deformação permanente das misturas. O confinamento variável no PTT simula melhor os confinamentos reais em pavimentos. Foi encontrada uma correlação mais forte dos resultados do teste PTT e do teste de rastreamento das rodas. Conclui-se que a adição de fibras de poliéster melhora o desempenho mecânico das misturas, e o método PTT é um novo método de teste confiável.
5 My memory	Os testes de laboratório foram realizados para avaliar os efeitos da fibra de poliéster na deformação permanente de misturas de asfalto. Os resultados indicam que as fibras melhoram a resistência à deformação das misturas. Os confinamentos no teste triaxial parcial (PTT) têm efeitos significativos sobre a deformação permanente das misturas. O confinamento variável no PTT simula melhor os confinamentos reais em pavimentos. Foi encontrada uma correlação mais forte dos resultados do teste PTT e do teste de rastreamento das rodas. Conclui-se que a adição de fibras de poliéster melhora o desempenho mecânico das misturas, e o método PTT é um novo método de teste confiável.
6 Promt	Os testes de laboratório conduziram-se para avaliar os efeitos da fibra de poliéster na deformação permanente de misturas de asfalto. Os resultados indicam que as fibras melhoram a resistência de deformação de misturas. Os confinamentos no teste de triaxial parcial (PTT) têm efeitos significantes sobre a tensão permanente de misturas. O confinamento variado em PTT melhor simula os confinamentos reais em pavimentos. Uma correlação mais forte de resultados de prova de PTT e o teste de rastreamento de roda encontrou-se. Conclui-se que a adição de fibras de poliéster melhora a realização mecânica de misturas, e o método de PTT é um novo método de experiência fiável.
7 Systran	Os testes laboratoriais foram feitos para avaliar os efeitos da fibra de poliéster na deformação permanente de misturas de asfalto. Os resultados indicam que as fibras melhoram a resistência à deformação das misturas. Os confinamentos no ensaio triaxial parcial (PTT) têm efeitos significativos na estirpe permanente das misturas. O confinamento variado no PTT simula melhor os confinamentos reais em pavimentação. Verificou-se uma correlação mais forte entre os resultados dos ensaios do PTT e o ensaio de seguimento das rodas. Conclui-se que a adição de fibras de poliéster melhora o desempenho mecânico das misturas e que o método PTT é um novo método de ensaio fiável.
8 Tradukka	Os testes laboratoriais foram realizados para avaliar os efeitos da fibra de poliéster na deformação permanente das misturas asfálticas. Os resultados indicam que as fibras melhoram a resistência à deformação das misturas. Os confinamentos no teste triaxial parcial (PTT) têm efeitos significativos sobre a tensão permanente das misturas. O confinamento variado no PTT simula melhor os confinamentos reais nas calçadas. Uma correlação mais forte dos resultados dos testes do PTT e do teste de rastreamento das rodas foi encontrada. Conclui-se que a adição de fibras de poliéster melhora o desempenho mecânico das misturas, e o método PTT é um novo método de teste confiável.
9 Wordlingo	Os testes de laboratório foram conduzidos para avaliar os efeitos da fibra do poliéster na deformação permanente de misturas do asfalto. Os resultados indicam que as fibras melhoram a resistência da deformação das misturas. Os confinamentos no teste triaxial parcial (PTT) têm efeitos significativos na tensão permanente das misturas. O confinamento variando no PTT simula mais melhor os confinamentos reais nos pavimentos. Uma correlação mais forte de resultados de teste do PTT e da roda que seguem o teste foi encontrada. Concli-se que a adição de fibras do poliéster melhora o desempenho mecânico das misturas, e o método do PTT é um método novo de confiança do teste.
10 Yandex	Os testes laboratoriais foram realizados para avaliar os efeitos da fibra de poliéster na deformação permanente de misturas de asfalto. Os resultados indicam que as fibras melhoram a resistência de deformação das misturas. As condições do ensaio triaxial parcial (PTT) têm efeitos significativos na estirpe permanente das misturas. O confinamento variável no PTT simula melhor a confinamento real nos pavimentos. Foi encontrada uma correlação mais forte dos resultados dos testes de PTT e do teste de rastreamento da roda. Conclui-se que a adição de fibras de poliéster melhora o desempenho mecânico das misturas, e o método PTT é um novo método de ensaio confiável.

When analyzing the strengths of MT systems, students may realize through Task 3 that Xu et al.’s (2014) abstract will be instantly machine translated in less than one second by the overall systems, indicating that MT technologies may help in boosting productivity in translation.

MT systems are relatively easy to use, where students only have to copy and paste the source text to be translated in the source text grid of the systems and to press a single button to have it translated in the target text grid.

All MT systems have output the translated text with the same punctuation marks and segmentation as in Xu et al.’s (2014) original abstract, in which sentences are presented in less than 25 words each, alluding to trends such as International or Global English for scholarly communication purposes (BOWKER; BUITRAGO CIRO, 2019).

As previously highlighted, although MT systems have output a quite sufficiently performance related to scientific terms of Xu et al.’s (2014) abstract - and, in the case of Babylon MT system that has left untranslated the word “confinements”, and Wordlingo MT system that has presented a spelling error in “concli-se” instead of “conclui-se” (to conclude), and an error related to the intensifier “mais melhor” (something like “more better”) -, their performance can be useful for informational purposes, or, as aforementioned, for assimilating the “basic message” (HUTCHINS, 2015). In this regard, it is worth noting that in terms of adequacy (MOORKENS, 2018), the overall systems have expressed the general or basic meaning of the source text in the target language grid, without adding awkward sentences.

Through Task 3, students are able to make informed decisions and choose between translating the abstract from scratch or working together with the machine, particularly in cases where the client does not provide specific guidance on how the work should be carried out.

4.4 Task 4 -Analyzing MT systems outputs and identifying their weaknesses

4.4.1 Sub-task: Major concerns

Before going through translational weaknesses of MT systems, students should be aware of the fact that their clients, as highlighted by Bowker and Buitrago Ciro (2019), may imagine that the data entered into a free online machine translation service simply disappear once the translation process is completed, but this is not true. On the contrary, machine translation service providers are typically interested in keeping these data for eventual reuse in the future (e.g., as training data for feeding systems) (BOWKER; BUITRAGO CIRO, 2019, p. 84). In Figure 4, for instance, when enabling a MT system in Wordfast Anywhere translation memory system, there is the following disclaimer (in the bottom of the MT setup screen): “Make sure MT complies with your confidentiality requirement because source segments are sent to the MT provider for translation”.

Figure 4
Machine Translation setup screen through the TMS Wordfast Anywhere

Accordingly, Task 4 aims at introducing students to the critical awareness (HURTADO ALBIR, 2008) of using MT systems. They need to decide if a determined innovative or top-secret research finding should be machine translated, and alert their clients, let alone copyright issues, and the fact that source texts, when sent via internet to online MT systems, can be intercepted by non-authorized or malicious people. Kenny and Doherty (2014) go further affirming that not only does free online machine translation may imply that translation is an agentless, an automatic function that can be done in no time at all (CRONIN 2012, p. 47 apud KENNY; DOHERTY, 2014, p. 288), but it can also obscure the human labor that produces the translated and other data on which MT is based. The authors also affirm that systems like Google Translate obscure even the labor of the computer scientists, who build MT systems.

Task 4 previews the discussion of another issue. When configuring MT systems, in just four of the 10 systems in Figure 2, students can choose Brazilian Portuguese rather than (European) Portuguese as target language, such as in Wordlingo, DeepL, Bing Microsoft, and Tradukka. When analyzing the translations, although there are no significant differences amongst the overall outputs in the case of Xu et al.’s (2014) abstract in hand, some systems may present certain characteristics of Portuguese variants. For the sake of illustration, Figure 5 displays the word “polemic” translated by DeepL MT system, firstly to European Portuguese “polémica”, where diacritics are wrongly used if Brazilian Portuguese is the target language. In the case of DeepL, students have to check the bottom of the screen to realize the correct use of the diacritics for Brazilian Portuguese translation of the word “polêmica”, both as a noun, or as an adjective.

Figure 5
The word “polemic” translated into European Portuguese by DeepL

4.4.2 Sub-task: Terminology

Before talking about MT’s weaknesses, it is worth noting that “terminology”, “word order”, “acronyms”, and “passive voice” analyzed below are categories herein used to be applied for our human evaluation of MT systems. That is to say, MT systems recognize probabilities, not “terminology”, “word order”, “acronyms”, or “passive voice”, but instead we use these categories to implement human evaluations of MT systems. The n-grams, a contiguous sequence of n items from a given sample of text are not linguistically driven, i.e., they do not correspond to categories in linguistic theory. In the case of statistical machine translation, Kenny and Doherty (2014) explain that:

What is most important for the interested translation scholar to note is, however, that in SMT the probability of a target language sentence is calculated based on the joint probabilities of the n-grams it contains. This type of modelling is called generative modelling and involves breaking a bigger problem up into a series of smaller steps. The n-grams themselves are not linguistically motivated, that is, they do not correspond to categories in linguistic theory like ‘constituent’ or even ‘collocation’. Sometimes n-grams are coterminous with constituents (the trigram ‘the biggest house’ coincides nicely with the constituent ‘noun phrase’ or ‘nominal group’, for example) but this is just a coincidence […] (KENNY; DOHERTY, 2014, p. 281)

As previously mentioned in section 2, the overall students have reported that a similar Civil Engineering research abstract to the one of Xu et al.’s (2014), has been quite sufficiently translated by MT systems, and three out of 25 have reported that before publishing the abstract into Portuguese or delivering it to the client, they should only control some terminological entries.

These three students were right when they refer to the need for checking and controlling terminological entries translated by MT systems when it comes to publish the translation or deliver it to the client. The terminology of a domain-specific text, such as the one displayed in the abstract at hand, poses challenges for machines, and even for humans. Therefore, appropriate research and documentation strategies should receive considerable attention for publication purposes.

Failures in using the appropriate terminology by the machine, as well as by the human translator cause significant issues during the translation process. Dunne (2007) describes how defective terminology can propagate errors in multilingual environments and can lead to mistranslation, defective target texts, and costly repercussions for the end user or client. In domain-specific translation courses aided by MT technologies, this seems to be the right moment to an increased focus on the role that human translators play in modifying machine translation outputs. It is the role of human translators to contextualize and manage the document lifecycle (DUNNE, 2007; MELLINGER, 2017).

In Xu et al’s (2014) abstract, the term “laboratory tests”, in the first sentence of the abstract “The laboratory tests were conducted to evaluate the effects of polyester fiber on permanent deformation of asphalt mixtures”, was machine translated by the 10 MT systems by “testes laboratoriais” ou “testes de laboratório” instead of “ensaios laboratoriais”, as it is used amongst the scholarly community who studies the topic (BORGES, 2014).

It can be argued amongst students that this common problem related to terminology, which could be found in all 10 MT systems, is due to the fact that translation renditions are likely to be proposed by the crowd, collaborative initiatives or machine translation providers, rather than by experts or higher status scientists or scholars (JIMÉNEZ-CRESPO, 2018; MOORKENS, CASTILHO, GASPARI, DOHERTY, 2018), what forces us -translators, teachers, and students to orbit around existing MT systems of general data mostly fed by the crowd. Therefore, terminology is an aspect of texts that all machine translation users will have to deal with.

In the sentence where we read “The varying confinement in the PTT better simulates the actual confinements in pavements”, although eight of the systems have output the term “pavimentos” as the Brazilian Portuguese translation for “pavements”, two of them have outperformed the translation “calçadas” (sidewalks), as it was the case of Bing MT and Tradukka MT systems. Though we can consider a sidewalk as a pavement, our world knowledge guides us to the conclusion that sidewalks are rarely paved with asphaltic mixtures and are not tightly pulled or stretched by heavy wheels, so excluding the translated term “calçadas” amongst the translation options for the term. Once more, this sub-task aims at highlighting the role of human translators to contextualize and manage terminology for publication purposes.

4.4.3 Sub-task: Word order in noun phrases

Beyond common terminological limitations, machine translation outputs reveal that word order tends to be generally perceived as difficult for machine translation systems. When linking elements are omitted from noun phrases, MT systems, and human translators alike, will seek for inferring meaning between the words.

In the sentence “… and the PTT method is a reliable new test method”, chances are that there will be problems when it is machine translated from English into Portuguese. The original meaning may be misinterpreted due to a literal translation as in “novo método de teste confiável”, instead of “um método novo e confiável para realização de ensaios” (new and reliable method for conducting tests). Through this task, students may realize that, where possible, they will have to split up machine translated noun phrases and clarify the relationships between words. It is worth noting that, according to Yamada (2019), MT systems of today, especially neural machine translation systems, produce human-like errors, becoming more difficult for students to recognize and correct them.

4.4.4 Sub-task: Acronyms

The original abstract contains the abbreviated form “PTT”, which stands for “partial triaxial test”. Comparing the MT outputs in Portuguese in Figure 3, students will observe and discuss in this fourth sub-task that MT systems have not translated the acronym. While it is very common to use acronyms and abbreviated terms in scientific papers, especially when authors want to avoid repetition and to save time when writing or reading them, they pose a translational problem, and communication will be confused rather than clarified. If the English acronym is well known by the scholarly community, then it should be maintained as in the original. But if this is not the case, when translating into Portuguese, students will investigate in Brazilian Portuguese literature and decide if “ETP”, what, in this case, would stand for “ensaio triaxial parcial” should be adopted or if the full term should be repeated. MT systems are commonly “blind” when it comes to acronyms and abbreviations.

4.4.5 Sub-task: Passive voice

In this sub-task, students may identify that passive voice is still one of the most common limitations across all MT systems. In the sentence “A stronger correlation of test results from the PTT and the wheel tracking test was found”, the person or object that has received the action comes first, and the agent is entirely omitted. All MT systems have kept the use of passive voice, either at the beginning or end of the sentence: “Uma correlação mais forte dos resultados do teste do PTT e roda do teste de rastreio foi encontrada.”, as in Wordlingo MT system; or “Foi encontrada uma correlação mais forte dos resultados dos testes de PTT e do teste de rastreamento da roda.”, as in Yandex MT system. Using the active voice means clearly identifying the agent who is performing the action. As a result, the sentence is more direct and easier to understand if the agent responsible for the action comes first, such as in “A pesquisa evidenciou uma correlação mais forte entre os resultados do ensaio …” (The research has demonstrated a stronger correlation of the test results…).

4.5 Task 5 -Reporting orally MT systems performance amongst peers

To be more effective in time, students may voice out to peers their findings in relation to MT strengths and weaknesses. Through Task 5, students may enlarge their MT judgments when socializing with peers.

4.6 Final Task -Building a self-evaluation report where students individually comment in a written form what they have learned during the task concerning MT systems possible usages, their different approaches and performance in relation to a determined translation task, and their strengths and weaknesses.

In the final task, when self-assessing what they have learned, students may write a three to five-page report answering the following questions (as in section 3.1): What are MT systems possible usages? Can MT systems of today and their different approaches perform differently in a particular translation task? Do MT systems present the same set of strengths and weaknesses? They can also close up the report prospecting other learning needs, posing an important question like: “Do MT systems present the same set of strengths and weaknesses with different text genres other than scientific paper abstracts?”

5. SUMMARIZING AND ENVISAGING OTHER DIDACTIC PROPOSALS

Conceived as a didactic proposal originated from a class research of naturalistic bias, this paper aimed at proposing possibilities, through a task-based learning, of incorporating a machine translation module in Translation programs at university level, with the purpose of educating Translation students on strengths and weaknesses MT systems have in common, reconfiguring students first perceptions and keeping them abreast of technological developments.

The task-based learning herein proposed, centered on competence-based approach, has the purpose to show that, for the time being, regardless MT advances, the 10 MT systems have shown some common strengths, such as the fast way TM systems accomplish a translation task, the possible boost in productivity, and the overall adequacy of the text for assimilation purposes. At the same time, they have shown some general and common limitations, such as the ones related to terminology, noun phrases, acronyms, and passive voice. Even with the advent of neural machine translation, which is the final word in machine translation research and tool, the same strengths and weaknesses are performed by MT systems of today, at least when it comes to the scientific paper abstract described above.

Contrary to the alleged early idea that MT has the purpose of replacing human translators and that its outputs are insufficient to deserve attention, they can fit the purpose of informing general professionals or users about the content of a message, and give translators the chance to decide to post-edit it or to start translating a text from scratch. Professional translators and students may decide assigning repetitive tasks to machines, and preserving their cognitive efforts to more complex problems of the text to be translated. What I think is the most powerful purpose of this sort of activity is that students should be reminded that they are in the center of the translation and educational processes. It is particularly in this sense that the words “teaching and learning” appear together in the title of this paper: this didactic proposal refers not only to the skills students are supposed to be taught, but also to what they can do with the acquired skills they have learned.

Concerning subsequent didactic proposals, other class activities may be planned in Translation programs in order to better inform and critically educate Translation students on the topic, such as activities involving different text genres other than scientific abstracts that may reveal not the same MT performances. Possibly applying a project-based teaching method, groups of students may be responsible for different text genre to enable comparisons amongst MT outputs. Studies on machine translated literary texts are insightful for MT teaching and learning, as highlighted by Toral and Way (2018).

Another didactic proposal could be planned dividing a class into two groups, where one group translates a text using MT systems, and the other group translates the same text without this technology. With this hands-on activity aided by a collaborative-based teaching method, students may estimate time and effort spent when translating a text with or without MT support, as well as formulating some practical post-editing strategies (ESQUEDA, 2019).

Additionally, with a problem-based teaching method in mind, teachers can suggest students to compare their (human) evaluations of MT outputs, in contrast to automatic evaluation of MT using systems such as BLEU (Bilingual Evaluation Understudy), an algorithm for evaluating the quality of texts which have been machine translated. The free Asiya-Online toolkit for Automatic Machine Translation Evaluation (Figure 6) can be used for MT output comparisons (http://asiya.lsi.upc.edu/demo/asiya_online.php).

Figure 6
The Asiya-Online main page

As Kenny and Doherty (2014) aptly put it:

As academics who train translators we have a vested interest in, but also an ethical commitment to ensuring the sustainability of the profession. Otherwise we are educating students in our own interest but with little regard for theirs. (We could, of course, stop educating translators altogether, but this would make little sense given our assessment of the current translation market […].) Such sustainability must partly reside in translators’ ability to evolve and to adopt whatever tools are useful and currently available to them so as to remain relevant and competitive. (KENNY; DOHERTY, 2014, p. 290)

In closing, even though I recognize, as a translator and translation teacher, that it is difficult to predict a technological future related to MT, and so to predict what should be learned by students, especially with the emergency of planning distance learning strategies due to coronavirus pandemic, for now I advocate that the merits and drawbacks of MT should be taught throughout a Translation curriculum, based on appropriate teaching approaches, methods and techniques, with authentic texts, available systems, and with correspondent assessment criteria, providing students the chance of not simply learn how and when to fix MT outputs, but how to intervene in MT workflows.

• 1
  The terms “strengths” and “weaknesses” were chosen based on Garcia (2010) and Moorkens (2018), but other terms such as “potentialities” and “limitations”, and “merits” and “drawbacks” will be sometimes used interchangeably in this paper.
• 2
  Naturalistic and interpretive studies are alternatives to positivist research, as understanding of individuals’ interpretations of the world around them has to come from the inside, not the outside (COHEN et al., 2018). A class research of naturalistic bias may take didactic proposals beyond speculation. This is not to say that approaching machine translation teaching by a previous qualitative and naturalistic class research is the only route for teachers. Nevertheless, it may help them to recognize students’ experiences and demystify social realities.
• 3
  Defining MT results as “quite sufficiently” students meant that MT systems have provided the “basic message” of the abstract.
• 4
  In view of its didactic nature, the content of this proposal may sound prescriptive. Yet, the sequence presented here is to my knowledge in accordance with students’ needs (see Section 2), but it is up to teachers to accept or reject it based on their educational motivations.

REFERENCES

• ARROJO, R. (1993). Tradução, Desconstrução e Psicanálise Rio de Janeiro: Imago.
• BORGES, R. L. (2014). Deformação permanente em misturas asfálticas a partir do Shift Model viscoplástico e ensaios triaxiais de carga repetida 2014. 194 f. Dissertação (Mestrado em Engenharia de Transportes) - Centro de Tecnologia, Universidade Federal do Ceará, Fortaleza. Retrieved from: http://repositorio.ufc.br/handle/riufc/11168 Latest access: March 20 2020.
  » http://repositorio.ufc.br/handle/riufc/11168
• BOWKER, L. (2020). Fit-for-purpose translation. In: O’HAGAN, M. The Routledge Handbook of Translation and Technology New York: Routledge, pp. 319-335.
• BOWKER, L.; BUITRAGO CIRO, J. (2019). Machine Translation and Global Research: Towards Improved Machine Translation Literacy in the Scholarly Community. United Kingdom: Emerald Publishing Limited.
• BOWKER, L.; MARSHMAN, E. (2010). Toward a Model of Active and Situated Learning in the Teaching of Computer-aided Translation: Introducing the CERTT Project. Journal of Translation Studies, v. 13, n.1-2, pp.199-226. Retrieved from: https://www.airitilibrary.com/Publication/alDetailedMesh?docid=P20180306001-201012-201803060012-201803060012-199-226 Latest access: March 16 2020.
  » https://www.airitilibrary.com/Publication/alDetailedMesh?docid=P20180306001-201012-201803060012-201803060012-199-226
• CATFORD, J. C. (1965). A Linguistic Theory of Translation London: Oxford University Press.
• COHEN, L. et al (2018). Research Methods in Education New York: Routledge .
• DOHERTY, S. et al. (2018). On Education and Training in Translation Quality Assessment. In: MOORKENS, J.; CASTILHO, S; GASPARI, F; DOHERTY, S. (eds.). Translation Quality Assessment: From Principles to Practice. Machine Translation Series. Switzerland: Springer International Publishing, pp. 95-106.
• DOHERTY, S; KENNY, D. (2014). The design and evaluation of a Statistical Machine Translation syllabus for translation students. The Interpreter and Translator Trainer, v. 8, n. 2, pp. 295-315. DOI: https://doi.org/10.1080/1750399X.2014.937571 Retrieved from: https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2014.937571?casa_token=AilV9T905zYAAAAA:kRVPcvz-SQIZ-heL8NoonR7weVkVTb2e5Ov4tbawk6q3G2oKz8HbPB4fbv1k6ZMifKGIJGRs3Pd1SHKd Latest access: April 22 2020.
  » https://doi.org/10.1080/1750399X.2014.937571» https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2014.937571?casa_token=AilV9T905zYAAAAA:kRVPcvz-SQIZ-heL8NoonR7weVkVTb2e5Ov4tbawk6q3G2oKz8HbPB4fbv1k6ZMifKGIJGRs3Pd1SHKd
• DELISLE, J. (2013). La traduction raisonnée: manuel d’initiation à la traduction professionnelle de l’anglais vers le français. 3^e édition. Ottawa: Les Presses de l’Université d’Ottawa.
• DUNNE, K. J. (2007). Terminology: ignore it at your peril. Multilingual, v. 87, n. 18, pp. 32-38.
• ERICSSON, K. A. (2000). Expertise in Interpreting: An Expert-Performance Perspective. Interpreting, v. 5, n. 2, pp. 187-220. DOI: https://doi.org/10.1075/intp.5.2.08eri
  » https://doi.org/10.1075/intp.5.2.08eri
• ESQUEDA, M. D. (ed.) (2020a). Ensino de Tradução: proposições didáticas à luz da competência tradutória. Uberlândia: Edufu. Retrieved from: http://repositorio.ufu.br/handle/123456789/30261 Latest access: March 14 2021.
  » http://repositorio.ufu.br/handle/123456789/30261
• ESQUEDA, M. D. (2020b). Ensino de tradução: Perfis e abordagens pedagógicas de docentes brasileiros e canadenses. Belas Infiéis, Brasília, Brasil, v. 9, n. 1, pp. 127-166. DOI: 10.26512/belasinfieis.v9.n1.2020.24657. Retrieved from: https://periodicos.unb.br/index.php/belasinfieis/article/view/24657 Latest access: March 14 2021.
  » https://doi.org/10.26512/belasinfieis.v9.n1.2020.24657» https://periodicos.unb.br/index.php/belasinfieis/article/view/24657
• ESQUEDA, M. D. (2019). Tecnologias da Tradução e Pedagogia Colaborativa. Tradterm, v. 34, pp. 48-80. DOI: https://doi.org/10.11606/issn.2317-9511.v34i0p48-80 Retrieved from: https://www.revistas.usp.br/tradterm/article/view/154784 Latest access: March 14 2021.
  » https://doi.org/10.11606/issn.2317-9511.v34i0p48-80» https://www.revistas.usp.br/tradterm/article/view/154784
• ESQUEDA, M. D. (2018). Ensino de Tradução: Culturas Pedagógicas. Trabalhos em Linguística Aplicada, Campinas, SP, v. 57, n. 2, pp. 1244-1273. Retrieved from: https://periodicos.sbu.unicamp.br/ojs/index.php/tla/article/view/8651880 Latest access: 14 mar. 2021.
  » https://periodicos.sbu.unicamp.br/ojs/index.php/tla/article/view/8651880
• FIOLA, M. (2003). Prolégomènes à une didactique de la traduction professionnelle. Meta, v. 48, n. 3, pp. 336-346. DOI: https://doi.org/10.7202/007594ar Retrieved from: https://www.erudit.org/en/journals/meta/2003-v48-n3-meta615/007594ar.pdf Latest access: April 2 2020.
  » https://doi.org/10.7202/007594ar» https://www.erudit.org/en/journals/meta/2003-v48-n3-meta615/007594ar.pdf
• FONSECA, N. B. DE L.; ALVES, F. (2016). Assessing complexity and difficulty levels of machine-translated texts. Letras & Letras, v. 32, n. 1, pp. 306-337. DOI: https://doi.org/10.14393/LL63-v32n1a2016-16 Retrieved from: http://www.seer.ufu.br/index.php/letraseletras/article/view/33248 Latest access: April 21 2020.
  » https://doi.org/10.14393/LL63-v32n1a2016-16» http://www.seer.ufu.br/index.php/letraseletras/article/view/33248
• GASPARI, F. (2001). Teaching machine translation to trainee translators: a survey of their knowledge and opinions. In: FORCADA, M. L., PÉREZ-ORTIZ, F., LEWIS, D. R. (eds.). Proceedings of the VIII Workshop Teaching Machine Translation, pp. 1-10. Retrieved from: http://mt-archive.info/MTS-2001-Gaspari.pdf Latest access: April 2 2020.
  » http://mt-archive.info/MTS-2001-Gaspari.pdf
• GILE, D. (2009). Basic concepts and models for interpreter and translator training Amsterdam, Philadelphia: John Benjamins.
• GONZÁLEZ-DAVIES, M. (2017). A Collaborative Pedagogy for Translation. In: VENUTI, L. (ed.). Teaching Translation: Programs, Courses, Pedagogies. New York: Routledge, pp. 71-78.
• GONZÁLEZ-DAVIES, M. (2004). Multiple voices in the translation classroom Amsterdam: John Benjamins.
• HEDBERG, L. E. (2019). Spanish-Swedish neural machine translation for the civil engineering domain 2019. 75 f. Final Thesis - Computer Systems and Languages, Universidad del País Vasco. Retrieved from: https://addi.ehu.es/handle/10810/34706 Latest access: April 16 2020.
  » https://addi.ehu.es/handle/10810/34706
• HURTADO ALBIR, A. (ed.). (2017). Researching Translation Competence by PACTE Group Amsterdam/Philadelphia: John Benjamins.
• HURTADO ALBIR, A. (2015). Aprender a traducir del francés al español: competencias y tareas para la iniciación a la traducción. Madrid; Castellón: Edelsa; Universitat Jaume I - Servicio de Publicaciones (Série Aprender a traducir).
• HURTADO ALBIR, A. (2008). Compétence en traduction et formation par compétences. TTR, v. 21, n. 1, pp. 17-64. DOI: https://doi.org/10.7202/029686ar Retrieved from: https://www.erudit.org/en/journals/ttr/2008-v21-n1-ttr2896/029686ar.pdf Latest access: April 4 2020.
  » https://doi.org/10.7202/029686ar» https://www.erudit.org/en/journals/ttr/2008-v21-n1-ttr2896/029686ar.pdf
• HUTCHINS, W. J. (2015). Machine translation: history of research and applications. In: SIN-WAI, C. (ed.). The Routledge encyclopedia of translation technology London/New York: Routledge, pp. 120-136.
• JIMÉNEZ-CRESPO, M. A. (2018). Crowdsourcing and Translation Quality: Novel Approaches in the Language Industry and Translation Studies. In: MOORKENS, J.; CASTILHO, S; GASPARI, F; DOHERTY, S. (eds.). Translation Quality Assessment: From Principles to Practice. Machine Translation Series . Switzerland: Springer International Publishing, pp. 69-93.
• KELLY, D. (2005). A handbook for translator trainers: a guide to reflective practice. Manchester, UK: St. Jerome.
• KENNY, D.; DOHERTY, S. (2014). Statistical machine translation in the translation curriculum: overcoming obstacles and empowering translators. The Interpreter and Translator Trainer , v. 8, n. 2, pp. 276-294. DOI: https://doi.org/10.1080/1750399X.2014.936112 Retrieved from: https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2014.936112?casa_token=N5ph6XsUy-4AAAAA:QFS5MTPuKHEs3XJprB2bhJr7qq8bFCnUfJc2fZeSqrz4a9mOf4X4Q72Zbm4ow1vNxPQ3uvfqlyDtZGiU Latest access: April 21 2020.
  » https://doi.org/10.1080/1750399X.2014.936112» https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2014.936112?casa_token=N5ph6XsUy-4AAAAA:QFS5MTPuKHEs3XJprB2bhJr7qq8bFCnUfJc2fZeSqrz4a9mOf4X4Q72Zbm4ow1vNxPQ3uvfqlyDtZGiU
• KIRALY, D. C. et al. (2016). Towards authentic experiential learning in translator education Mainz University Press: V & R Unipress.
• KIRALY, D. C. (2000). A social constructivist approach to translator education: empowerment from theory to practice. Manchester: St. Jerome.
• MARTINS, R. T.; NUNES, M. G. (2005). Noções gerais de tradução automática. Notas didáticas do ICMC - USP, São Carlos, n. 68, pp.1-26. Retrieved from: http://www.nilc.icmc.usp.br/nilc/download/NotasDidaticasICMC_68.pdf Latest access: April 2 2020.
  » http://www.nilc.icmc.usp.br/nilc/download/NotasDidaticasICMC_68.pdf
• MELBY, A. K. (2020). Future of machine translation: Musings on Weaver’s memo. In: O’HAGAN, M. The Routledge Handbook of Translation and Technology . New York: Routledge, pp. 419-436.
• MELLINGER, C. D. (2017). Translators and machine translation: knowledge and skills gaps in translator pedagogy. The Interpreter and Translator Trainer , v. 11, n. 4, pp. 1-15. DOI: https://doi.org/10.1080/1750399X.2017.1359760 Retrieved from: https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2017.1359760?needAccess=true Latest access: April 4 2020.
  » https://doi.org/10.1080/1750399X.2017.1359760» https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2017.1359760?needAccess=true
• MELO, G. F. (2018). Pedagogia Universitária: aprender a profissão, profissionalizar a Docência. Curitiba: Editora CRV.
• MOORKENS, J.; CASTILHO, S; GASPARI, F; DOHERTY, S. (eds.). (2018). Translation Quality Assessment: From Principles to Practice. Machine Translation Series . Switzerland: Springer International Publishing.
• MOORKENS, J. (2018). What to expect from Neural Machine Translation: a practical in-class translation evaluation exercise. The Interpreter and Translator Trainer , v. 12, n. 4, pp. 375-387. DOI: https://doi.org/10.1080/1750399X.2018.1501639 Retrieved from: https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2018.1501639?casa_token=ASt3VlZk6dAAAAAA:mqNqBzb2beUjVblrjBzegHk-HACgjunD3ZDj43ualQand-tZMZAD08iuU7ylq4Mqubk3N2aCCGlkE33e Latest access: April 21 2020.
  » https://doi.org/10.1080/1750399X.2018.1501639» https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2018.1501639?casa_token=ASt3VlZk6dAAAAAA:mqNqBzb2beUjVblrjBzegHk-HACgjunD3ZDj43ualQand-tZMZAD08iuU7ylq4Mqubk3N2aCCGlkE33e
• MOUNIN, G. (1975). Os Problemas Teóricos da Tradução Tradução de Heloysa de Lima Dantas. São Paulo: Cultrix.
• NORD, C. (2016). Análise textual em tradução: bases teóricas, métodos e aplicação didática Tradução e adaptação de Meta Elisabeth Zipser. São Paulo: Rafael Copetti Editor.
• O’BRIEN, S. (2002). Teaching post-editing: a proposal for course content. _{^{Proceedings of the 6th EAMT Workshop Teaching Machine Translation}} , pp. 99-106. Retrieved from: https://www.researchgate.net/publication/228787033_Teaching_Post-diting_A_Proposal_for_Course_Content Latest access: March 27 2020.
  » https://www.researchgate.net/publication/228787033_Teaching_Post-diting_A_Proposal_for_Course_Content
• OLOHAN, M. (2016). Scientific and Technical Translation London and New York: Routledge.
• PIMENTA, S. G. (1995). O Estágio na formação de professores: unidade entre teoria e prática. Cadernos de Pesquisa, n. 94, pp. 58-73. Retrieved from: http://publicacoes.fcc.org.br/ojs/index.php/cp/article/view/839/845 Latest access: April 4 2020.
  » http://publicacoes.fcc.org.br/ojs/index.php/cp/article/view/839/845
• PYM, A. (2013). Translation Skill-Sets in a Machine-Translation Age. Meta, v. 58, n. 3, pp. 487-503. DOI: https://doi.org/10.7202/1025047ar Retrieved from: https://www.erudit.org/en/journals/meta/2013-v58-n3-meta01406/1025047ar.pdf Latest access: April 1 2020.
  » https://doi.org/10.7202/1025047ar» https://www.erudit.org/en/journals/meta/2013-v58-n3-meta01406/1025047ar.pdf
• RISKU, H. (2016). Situated learning in translation research training: academic research as a reflection of practice. The Interpreter and Translator Trainer , v. 10, n. 1, pp.12-28. DOI: https://doi.org/10.1080/1750399X.2016.1154340 Retrieved from: https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2016.1154340?needAccess=true Latest access: April 25 2020.
  » https://doi.org/10.1080/1750399X.2016.1154340» https://www.tandfonline.com/doi/pdf/10.1080/1750399X.2016.1154340?needAccess=true
• ROBINSON, D. (2002). Construindo o tradutor Tradução de Jussara Simões. Bauru: EDUSC.
• SAVIANI, D. (2007). Pedagogia: O espaço da educação na universidade. Cadernos de Pesquisa , v. 37, n. 130, pp. 99-134. DOI: https://doi.org/10.1590/S0100-15742007000100006 Retrieved from: http://www.scielo.br/pdf/cp/v37n130/06.pdf Latest access: April 4 2020.
  » https://doi.org/10.1590/S0100-15742007000100006» http://www.scielo.br/pdf/cp/v37n130/06.pdf
• SHIWEN, Y; XIAOJING, B. (2015). Rule-based machine translation. In: SIN-WAI, C. (ed.). The Routledge encyclopedia of translation technology . London/New York: Routledge, pp. 186-200.
• TORAL, A.; WAY, A. (2018). What Level of Quality Can Neural Machine Translation Attain on Literary Text? In: MOORKENS, J.; CASTILHO, S; GASPARI, F; DOHERTY, S. (eds.). Translation Quality Assessment: From Principles to Practice. Machine Translation Series . Switzerland: Springer International Publishing, pp. 263-287.
• VIEIRA, L. N. (2020). Post-editing of machine translation. In: O’HAGAN, M. The Routledge Handbook of Translation and Technology . New York: Routledge, pp. 319-335.
• WAY, A. (2018). Quality Expectations of Machine Translation. In: MOORKENS, J.; CASTILHO, S; GASPARI, F; DOHERTY, S. (eds.). Translation Quality Assessment: From Principles to Practice. Machine Translation Series . Switzerland: Springer International Publishing, pp. 159-178.
• XU, T. et al. (2014). Evaluation of permanent deformation of asphalt mixtures using different laboratory performance tests. Construction and Building Materials, n. 53, pp. 561-567. DOI: https://doi.org/10.1016/j.conbuildmat.2013.12.015 Retrieved from: https://www.sciencedirect.com/science/article/pii/S0950061813011719casa_token=MQtIzB4AuAEAAAAA:S63gqInpJYSouY_7N1AVeLK44D5mO19j8Q059hlCyREwSOBRK9o8XhDpvkS0eVm3ch-i2QZNWjM Latest access: April 1 2020.
  » https://doi.org/10.1016/j.conbuildmat.2013.12.015» https://www.sciencedirect.com/science/article/pii/S0950061813011719casa_token=MQtIzB4AuAEAAAAA:S63gqInpJYSouY_7N1AVeLK44D5mO19j8Q059hlCyREwSOBRK9o8XhDpvkS0eVm3ch-i2QZNWjM
• YAMADA, M. (2019). The impact of Google neural machine translation on post-editing by student translators. The Journal of Specialised Translation, v. 31, pp. 87-106. Retrieved from: http://www.jostrans.org/issue31/art_yamada.pdf Latest access: April 3 2020.
  » http://www.jostrans.org/issue31/art_yamada.pdf
• ZETZSCHE, J. (2016). LILT Translation environment tool of a different kind. Multilingual , Sandpoint, Idaho: Multilingual Computing Inc., v. 27, n. 1, pp. 15-17.

Publication Dates

• Publication in this collection
  14 May 2021
• Date of issue
  Jan-Apr 2021

History

• Received
  25 June 2020
• Accepted
  11 Mar 2021
• Published
  15 Mar 2021