Post-Editing as the Means to Activate Students’ Thinking and Analytical Process: Psycholinguistic Aspects Пост-редагування машинного перекладу як засіб активізації мисленнєво-аналітичної діяльності студентів: психолінгвістичний аспект

The aim of the research is looking for the ways to intensify the future translators’ analytical and thinking activity during their independent work in online teaching. The author strives to achieve it through the combination of post-editing machine-translated texts and the Think-aloud protocol procedure. It is also assumed that this combination Пост-редагування машинного перекладу як засіб активізації.. . 222 ©  Черноватий Леонід & Ковальчук Наталя reduces the students’ dependence on the MT target text structure, as well as improves their competence in translating specialized texts. The methodology of research involved experimental post-editing-based online teaching (28 contact hours and 92 hours of independent work) of an elective university course ‘Specifics of translating English-language discourse in the domain of Psychology’ to the first-year MA students (majoring in English and Translation) whose command of English ranged from C1 to C2 levels in the CEFR classification. The parameters of analysis included the percentage of the students’ uploaded home tasks, the degree of the subjects’ post-editing intensity in their weekly homework, the students’ independence in the interim and final tests, as well as the marks in the Final test. The results of the analysis demonstrated a substantial difference between various groups of the subjects by all indicators. The amount of home tasks uploaded by the subjects in groups A and B (with more intensive analytical and thinking activity) exceeds the similar parameter in groups C and D (with less intensive activity) more than twofold. There is a considerable advantage of the groups A and B (and even C) subjects’ post-editing intensity in their weekly homework as compared to group D. The intensity of the students’ analytical and thinking activity decreased from the highest (group A) to moderately high (group B) to average (group C) and to low (group D). The degree of the students’ independence in the interim and final tests decreased from 85.0% (group A) to 35.0% in group D, with the remaining groups’ indicators in between – 59.0% (group B) and 46.0% (group C). These indicators clearly correlate with the average marks in the final test, which amounted to 93.80, 63 and 53 points (out of 100) in groups А, В, C and D respectively. Conclusions. Post-editing, in combination with the modified TAP procedure, contributes to the efficient development of the specialized texts translation competence due to the intensification of the students’ analytical and thinking activity, reduces their dependence on the MT target text structure and correlates with the improvement of the overall quality of their translation.


Introduction
Before the actual presentation of our research, it seems reasonable to define some of the terms used in this paper to avoid their misinterpretation. They include: human translation -the one done by a human being; trans-human translation -the one that goes beyond the purely human translation and additionally involves automated translation systems, such as machine translation (MT), computer aided translation (CAT) tools, translation memory (TM), etc.; pre-editing -removing from the source text (ST) specific linguistic features (or negative translatability indicators (Underwood & Jongejun, 2001) that may be problematic for MT systems (Mellinger, 2018); post-editing -may range from editing only the MT target text (TT) to a complete comparison of the ST and TT with revisions made for style, terminology, or logic (Mellinger, 2018); editing (review) -involves only the TT and is not checking the TT against the ST (ISO 171000, 2015;Mellinger, 2018); revision -constitutes a check of the TT against the ST (ISO 171000, 2015;Mellinger, 2018); Think-aloud Protocol (TAP) -a data-gathering method where the subjects verbalise their thought process (Cifuentes-Ferez & Rojo 2015).
The research into the impact of information and communication technologies (ICT) upon the psychological processes in the students' minds in translator and interpreter training and teaching, and consequently -on the efficiency of the latter, has been in the focus of many authors' attention and the subject of numerous publications.
Specifically, a large-scale survey of 438 validated respondents, including freelance translators, language service providers, translator trainers, and academics (Gaspari et al., 2015), proved the ever increasing importance of ICT, including MT, assessment techniques and postediting in the translation and localisation industry. A similar survey of translation technology adoption (Man et al., 2020) among 441 Chinese students enrolled on a MA Programme in Translation and Interpreting demonstrated a strong, positive correlation between students' knowledge and their use of translation technology. Other authors (Kenny & Doherty, 2014;Rodríguez de Céspedes, 2019) also support the idea of designing and implementing a new translator-oriented MT syllabus, arguing for a holistic approach to the integration of MT into translator training programmes, one that empowers rather than marginalises translators.
The comparative efficiency of human and post-edited MT remains a contested issue. Some authors (Loock, 2018), basing on their original research, argue for an unconditional advantage of human over machine translation. Other researchers (Daems et al., 2017), also basing on the original experiments, came to the conclusion that both methods are more similar than anticipated. Still others (Yang et al., 2021) report that MT post-editing is more efficient than human translation while the research into the Korean undergraduate students' post-editing the output of Google Translate (medical texts) (Sang-Bin, 2018) found the procedure to be inefficient. The students, among other things, inadequately performed pre-and post-editing, used dictionaries too frequently and spent too much time on non-technical words. The author concludes that the priority should be given to improving students' basic translation competence rather than skills specific to post-editing.
There have been attempts to rethink the opposition of human translation vs ICT (including MT) translation (Alonso & Calvo, 2015), suggesting the idea of trans-human translation that goes beyond the purely human translation and additionally involves automated translation systems. The authors believe the idea envisages a stimulating future for translators, where they will use technological extensions in a creative and critical way.
The arrival of next-generation MT software, including the one based on statistical and neural networks (Schmidhofer & Mair, 2019;O'Brien & Rossetti, 2020), made some authors (Pym, 2013) predict its eventual replacement of fully human translation in many spheres of activity. Thus, it is suggested (Pym, 2013;Mellinger, 2017) to incorporate MT in multiple courses across the curriculum (or, at least, the majority of translation-related disciplines), rather than concentrating the material in a stand-alone course or module, though there are some other opinions concerning this issue as well (Esqueda, 2021).
On the other hand, the research (Moorkens, 2018) into comparative efficiency of neural (NMT) and statistical machine translation (SMT) systems in several language pairs proved that the euphoria concerning the new MT engines is slightly premature.
In general, though most of the authors agree that MT is one of the most innovative technologies in the field of translation, many of them have shown certain restraint, arguing that it is still too early to predict how it can support the creativity and productivity of professional translators in the future (Burchardt et al., 2016).
The problem of MT application in the future translator training and teaching has gone complicated after this process went completely online because of the COVID restrictions. As all home tasks (at least those to be done in writing) within this specialization involve translation, and each student can immediately translate any text by just touching a button, there emerge serious concerns about the possibility of developing a full-fledged translator competence under such conditions. During this momentary operation, the student's mind remains passive, as it is not engaged either in the ST processing, translation problems identification and the related decision making concerning many levels (semantic, lexical, grammatical, stylistic, logical, etc.) involved, or the TT generation. The said touch on the button does not contribute to developing a corresponding neural network that provides for the translation process in the student's mind. Thus, the translator competence development just does not occur. The teacher's calls for the MT-free homework are generally ineffective because the temptation to save time for most of the students is too strong to resist. It has been proven experimentally as well: one research (Yang et al., 2021) found that the students perceive the amount of time and work that is saved when using MT to be a greater benefit than the overall utility of MT. The expectations that students compensate the said loss in the amount of analytical and thinking activity by post-editing the MT TT rarely come true. Most students tend to limit their operations to the mere replacement of several words in a sentence with their synonyms, if any.
The direct ban on the use of MT in home tasks is counterproductive either -the students easily avoid it, for example, by involving a third language. Say, the machine translates the ST from English into German and then -from German into Ukrainian. Teachers would find it more difficult to prove the MT manipulations in this case. Thus, the aim of our research was to develop a translator training system that could promote the students' analytical and thinking activity in the process of doing their home tasks in the hope that it would contribute to the efficient development of each student's translator competence. To achieve this aim, we had to accomplish several objectives: to define the principles of the said system; select the subjects and the domain the research would be related to; organize and conduct the experimental teaching and tests; represent, process and interpret the experimental data; formulate the preliminary conclusions; outline the prospect of further research.
The absence of the research into the ways of boosting the students' analytical and thinking activity and its impact on the efficiency of online instruction provides for the novelty of this study, because the said form of teaching has never been used on such a large scale.

Methodology
Our research was based on the following hypothesis: In online translation teaching, the regular post-editing of MT TT, in combination with the TAP procedure, promotes the students' analytical and thinking activity in their independent work. In its turn, the said activity contributes to the efficiency of specialized texts translation skills development. It is expected that, as a result, the students' dependence on the MT TT structure will be reduced, while the quality of their independent (without MT) translation will improve.
Because, as it was mentioned earlier, the ban on the MT use in the students' home tasks is not productive, within our approach, the MT application was not only allowed, but it was mandatory. That is, the students had to translate the ST from English into Ukrainian by means of the Google Translate system and post-edit the resulting MT TT. To promote the students' analytical and thinking activity in doing their home tasks, and avoid the formalistic approach to their post-editing (mere substitution of several words with their synonyms, etc.), we used a modified form of the Think-aloud protocol (TAP) (Cifuentes-Ferez & Rojo, 2015) experimental procedure. In it, unlike the TAP classical form, where the subjects' reflections on the actions they perform are recorded and then analyzed, the students had to explain in writing, why they had edited the particular fragment of the MT TT (see the example below) -the fragments changed in the process of post-editing are shown in italics. ST MT TT Post-edited TT It has been argued that due to the nature of the problem of consciousness, empirical tests are intrinsically impossible.
Concerning the post-editing of this sentence, the students' explanations could be as follows: In the first sentence, at the syntactic level, we see practically completely literal translation -the words in the MT TT almost exactly follow their order in the ST. Consequently, the MT TT looks unnatural and requires some rephrasing and word order changes. The literal translation prevails at the word and collocation levels as well (it has been argued -було аргументовано, nature of the problem of consciousness -природа проблеми свідомості, empirical tests are intrinsically impossible -емпіричні тести по суті неможливі). Thus, at the post-editing stage, the target text collocations were modified by means of contextual substitution (вважається, специфіка цієї проблеми, робить застосування емпіричних тестів… практично неможливим) to meet the target language norms. Other changes in the post-edited text included word additions: "робить застосування" and "при її дослідженні".
To test our hypothesis, we conducted a pilot experimental study within a semester-long elective course 'Specifics of translating English-language discourse in the domain of Psychology'. Its duration constituted 120 academic hours (28 contact hours and 92 hours for independent work). The course covered some basic topics of Psychology as a science (History of Psychology, Methods of Research, Cognitive Psychology, Consciousness, Intelligence, Attention, Perception, Thinking, Personality and Memory) and was based on the appropriately orientated original texts.
The subjects included 23 first-year (second semester) MA students majoring in Germanic languages and Literature (Translation included) (20 females and 3 males) of the School of Foreign Languages at V.N. Karazin Kharkiv National University (Ukraine), 21 to 24 years of age, who studied English as their first foreign language. By the beginning of the experiment, their total amount of contact hours for learning English at university exceeded 1500, without taking into account their related experience in the secondary school. Thus, within the CEFR classification (CEFR, 2018), the expected level of their command of English ranged from C1 to C2. Their experience in specialized translation (social-political, economic, legal, in the domains of technology and natural sciences) amounted to over 200 contact hours. The subjects were properly motivated as the results of their homework were taken into account when assessing their progress within the said course.
Because of the previously mentioned COVID restrictions, this course was conducted completely online. Once a week, the students had one online class (80 minutes) with a teacher, where a new topic material was introduced, and the students did some sight translation exercises concerning the texts the teacher displayed on the screen. Then the students were given their home task, which included some preparatory exercises, as well as the texts, which the students translated by means of MT and then post-edited in the above-mentioned manner. The results of their post-editing, complete with their explanation of the accompanying reasons for their corrections, had to be uploaded into a corresponding Google Classroom before the beginning of the next class.
To stimulate the students' work, we used a system that enabled each of them to be awarded 60 points for their semester activities (14for attendance (1 х 14), 18 -for home-tasks (1,5 х 11 plus 1,5 bonus for the upload of all home tasks), 6 -for online class presentations of their post-editing (3 х 2), 22 -for interim tests (11 х 2) and 40for the final test.
The weekly home tasks uploaded by the students into their Google Classroom were analysed according to several indicators. This paper is limited to the investigation of only one of them -the degree of the students' post-editing intensity. As it was mentioned above, it is critical to prevent the students' formalistic approach to the post-editing activities and provide for their creativity in this process, thus contributing to the promotion of their analytical and thinking activity.
To establish the degree of the students' post-editing intensity, which, as we had assumed, had to contribute to the translation efficiency in general, we analysed each student's work according to a number of criteria (see further) and, following the results of the analysis, referred it to one of the four levels (A -D): A -completely independent post-editing, many corrections with motivated explanations; B -mostly independent post-editing, a moderate quantity of corrections, occasionally lacking motivated explanations; C -mostly formal post-editing, an insignificant amount of corrections, mostly lacking motivated explanations; D -completely formal post-editing, few corrections without motivated explanations, omission of obvious errors in the MT TT.
To fine-tune the process of the students' home tasks classification, we developed a special 10-point scale, where each of the four levels was assigned a numeric indicator (see Table 1).

Table 1
The assessment levels and their numeric indicators If, having checked the student's home task, the teacher decides that by the degree of the post-editing intensity (see the meaning of this indicator above) this particular work should be undoubtedly classified as belonging to level A, s/he gives it the mark of 10. If the teacher hesitates between the levels of A and B, and finally decides in favour of A, the mark is 9, but if s/he chose B, the mark is 8. However, if the teacher does not have any doubts that it is level B, the mark is 7. The same procedure is applied if the student's work belongs to one of the lower levels C or D, or their combinations.
To provide the data for the analysis and establish the hypothetical impact of students' analytical and thinking activity intensity in doing their homework upon the change of their attitude to post-editing, as well as on the expected improvement of their translation quality, we carried out two interim tests and a final one. Formally, within those tests, the students were not allowed to use MT engines under threat of cancellation, but there were no technical means to exclude this possibility, so the students' TTs were evaluated by comparing them with the MT TT. The interim tests were completed at the fourth and seventh class periods, while the final test -at the last one. During the said tests the students translated from English into Ukrainian the texts related to the topics they had covered in the preceding course segment (Interim test 1 -The cognitive model of mind, Interim test 2 -The relationship between the intelligence and attention, Final test -Psychological and neural conceptions of memory). The text size and the time allowed for translation amounted to 1369 characters and 40 minutes in Interim test 1, 2803 characters and 70 minutes in Interim test 2, and 2958 characters and 70 minutes in the Final test, The interim and final tests were assessed by a number of indicators, from among which, in this paper, we analyse only the degree of the students' post-editing intensity in comparison with the overall accuracy of rendering the ST information in the TT. In the latter case, three types of errors and mistakes were differentiated with the corresponding amount of penalty points (p.p.) assigned for each of them: type 1 -language mistakes that do not prevent the comprehension of the ST content (0.1 p.p.); type 2 -ambiguity that (depending on the target readers' background and/or subject knowledge) might prevent the comprehension of the ST content (0.5 p.p.); type 3 -omission or distortion of the ST content that causes its misunderstanding on the part of the target reader (1.0 p.p.). The sum of each student's penalty points was then transformed into a conventional 100-point scale by means of a specially developed conversion scale.

Adherence to Ethical Standards
Before the actual experiment, we obtained informed consent from its potential participants to take part in the research. In the course of our empirical research, we observed the applicable ethical principles, specifically the principles of voluntary participation, informed consent and confidentiality. Because the experiment was a natural one, no funding was received to conduct this research. The authors do not have any financial or property interests concerning the materials presented in this article.

Results and discussion
In this section, we present our results concerning the degrees of the students' post-editing intensity in their weekly home tasks and their independence in the Interim and Final tests.
The data underlying this article are available in the international repository Social Science Research Network (Chernovaty, 2021). Table 2 shows the results of the subjects' performance analysis within cycle 1 -home tasks (weeks 1-3) and the interim test 1 (due to technical problems, we excluded two subjects -N7 and N9 -from our analysis).
The equally obvious negative correlation (that also corroborates our assumption) is clear concerning the subjects who did not do any home tasks at all (subjects 2, 3 and 18), did them formally (13 and 19) or partially and mostly formally (6,11,20). The degree of their independence in the Interim test 1 was very low (1-3 points), i.e. they uploaded the non-edited MT TTs, or their changes were purely symbolic (addition or deletion of some of the words, synonyms substitution with corresponding grammar agreements without any essential changes in the sentence structure or its lexical composition). For example, the sentence Our thoughts can then serve as the basis of "outputs", such as language or physical behavior, which was translated by the computer as Тоді наші думки можуть слугувати основою "результатів", таких як мова чи фізична поведінка, was completely preserved by subject 3, who just substituted the first and the fifth words with their synonyms (the substitutions are shown in bold type): Таким чином, наші думки можуть бути основою 'результатів', таких як мова чи фізична поведінка.
On the other hand, the results of some subjects (8, 15 and 22) in the Interim test 1 do not meet the expectations that followed from their home task indicators. The most striking difference was observed concerning subjects 8 and 15, who demonstrated very high results in their weekly home tasks (9-10 points in each of them) but performed relatively worse in the Interim test 1 (6 and 4 points correspondingly). Thus, the TT (otherwise of sufficiently good quality) of subject 8, contained several rather long fragments that literally coincided with the MT TT. Their length amounted to 9, 10 (two fragments) and 15 words, which definitely proves the impact of the MT TT. The same is true about the TT of subject 15, where there is a considerable amount of literal coincidences (5-6 words long) with MT TT. In-between these fragments there might have been some words different from the MT TT, but then this pattern repeated again. This sandwich-like TT looks more like a post-edited MT TT than an original translation. Table 3 shows the results of the subjects' performance analysis within cycle 2 -home tasks (weeks 4-6) and the interim test 2. Table 3 The degrees of the students' post-editing intensity in their weekly home tasks 4-6 and their independence in the Interim test 2 (TN -task number, Ʃ -sum total, IT -interim test, (-) -the task was not uploaded by the student) TN Subjects 1 2 3 4 5 6 8 9 11 12 13 14 15 16 17 18 19 20 21 22 23 4 10 6 1 10 7 -10 9 -10 -9 10 7 10 ---10 9 10 5 10 -1 9 7 -9 10 8 10 2 -9 8 8 -3 -10 9 10 6 10 7 1 10 8 -10 9 --1 10 9 8 10 -1 -10 7 10 Ʃ 30 13 1 29 22 -29 28 8 20 3 19 28 23 28 -4 -30 25 30 IT-2 10 3 2 8 6 -7 10 4 -4 7 8 7 8 7 3 -10 7 9 As it is clear from table 3, in cycle 2, if we discard the results of two subjects, who either did not participate in this cycle at all (subject 6) or did not write the Interim test 2 (subject 20), there is a general improvement of the subjects' performance. This upgrading concerns both the home task indicators (except subjects 12 and 19) and the Interim test 2, where the share of the subjects' results in the range of 7-10 points increased more than twofold (61.0% against 30.0% in the Interim test 1). Only the results of subjects 2, 3, 13 and 19, though slightly higher, remained at a low level.
In transposing the fragments of the imperfect MT-generated sentence, the combined effect of incomplete understanding of its content and its insufficient comparison with the ST resulted in semantic deviations (up to the use of antonymic expressions) and the preservation of identical fragments in different segments of the same sentence. For instance, the MT TT of sentence 5 (Therefore, we used, as a measure of the control of attention, the benefit for recall when a channel was monitored as opposed to not monitored in the dual-task situation) may be divided into six fragments: Тому ми використали (А) як міру контролю уваги (В) вигоду для відкликання (С), коли канал відстежувався (D), на відміну від відстежуваного (E) в ситуації подвійного завдання (F). Subject 3 transposed these fragments without any substantial changes: Тому коли канал відстежувався (D), ми використовували (А) перевагу згадування (C), як вимір контролю уваги (B), коли канал спостерігався (D), на відміну від того, який відстежується (E) в ситуації подвійного завдання (F). In fronting element D, she forgot to delete it in its former position, and thus it appears in this sentence twice -in the first and the fourth positions (see above). The fact that the subject relied on the MT TT is also proven by the fact that she retained in her TT the global error made by the machine -the computer translated the not monitored (channel) as (канал) який відстежується (monitored channel). Table 4 shows the results of the subjects' performance analysis within cycle 3 -home tasks 7-11 and the Final test.
On the other hand, subject 20, who had not upload any home tasks within this cycle, achieved a high result (8 points) in her Final test.
Thus, we may assume a certain correlation between the amount of quality post-editing MT-generated TT during the subjects' homework and the degree of their independence (from the structure of the MT TT) in translating specialized texts. To establish the impact of the said amount on the general quality of the subjects' translation, we arbitrarily divided them into four groups (A-D) according to the degrees of their post-editing intensity in their weekly home tasks and their independence in the Interim and Final tests (the explanation of the meaning of levels A-D is provided above, before table 1). Table 5 shows the comparative data related to the said groups. As it can be seen in table 5, there are considerable differences between the groups by all parameters. The amount of home tasks uploaded by the subjects in groups A and B exceeds the similar parameter in groups C and D more than twofold (group A) and almost twofold (group B). Moreover, there is a considerable advantage of the groups A and B (and even C) subjects' post-editing intensity in their weekly home tasks as compared to group D (98.0%, 83.0% and 78.0% in groups A, B and C respectively against the mere 35.0% -in group D). It generally means that throughout the semester, the intensity of analytical and thinking activity in the process of doing their home tasks was the highest one in group A (98.0%), moderately high (83.0%) -in group B, average (78.0%) -in group C, and low (35.0%) -in group D, It is corroborated by the subjects' independence in the Interim and Final tests, the degree of which consistently decreases from 85.0% in group A до 59.0%, 46.0% and 35.0% in groups B, C and D respectively. These tendencies correlate well with the performance of each group subjects in the Final test, where the average marks amounted to 93, 80, 63 і 53 points (out of 100) in groups A, B, C and D respectively.

Conclusions
Our pilot study was based on the hypothesis that the regular (weekly home tasks throughout a semester) post-editing of TT translated by means of the MT (Google Translate) system in combination with the TAP (Think-aloud protocol) procedure, modified for written translation, would contribute to the efficient development of specialized texts translation skills due to the promotion of students' analytical and thinking activity. The experimental results proved that an appropriate amount of the said post-editing reduced the students' dependence on the MT TT structure and correlated well with the quality of their independent (without MT) translation.
Our research also proved that special efforts are required to convince all the students of the necessity to do their post-editing tasks responsibly in order to provide for the proper amount and intensity of their analytical and thinking activity, and to prevent the students' formalistic approach to post-editing (sporadic substitution of some words in the MT TT with their synonyms, mechanic transposition of the MT TT sentence fragments, lack of proper argumentation of the MT TT corrections made by the student, etc.). The formalistic approach deprives post-editing of its teaching effect as, in this case, a proper intensity of the students' analytical and thinking activity is not achieved, and its potential positive impact upon the development of the students' competence of translating specialized texts is substantially diminished.
To test the preliminary conclusions suggested in this paper, the experiment ought to be replicated in another educational context on a different material and involving a larger sample of subjects. In our experiment, the subjects were compared with each other, and we also contrastively assessed the indicators of arbitrary groups (A, B, C and D). However, the prospective future research should use the control and experimental groups instead, to improve the reliability of conclusions. This assumption requires additional verification, which is the prospect of our further investigation.