Current assessment practices in Ukrainian translation classroom: teachers’ survey results

Висновки. Результати цього дослідження дозволили нам скласти агрегований профіль українських викладачів перекладу з точки зору практики оцінювання, визначити найпоширеніші способи проведення моніторингу, межового та підсумкового оцінювання під час навчання перекладу студентів бакалаврату в Україні та стверджувати, що суб'єктивна оцінка ефективності оцінювання викладачів перекладу залежить від їх педагогічного досвіду.

Introduction

Over the last few decades, translation teaching methodology has experienced sufficient progress in terms of the development of translation competence structure and its acquisition models, selection of relevant approaches to translators' training, design of appropriate methods and tools for teaching different types of translation and interpretation. However, assessment, being one of the key components of any training system, has not gotten the attention it really deserves in the context of translators' training.

According to Yan, Pan & Wang (2018), only 10% of 323 research articles, devoted to the issues of translation training and published by 10 leading specialised journals from 2000 to 2012, concerned some particular aspects of both academic and professional translation assessment. This fact is supported by meta-analysis results of 208 publications in 11 translation-oriented journals in the period of 2006-2015 carried out by Abdel Latif (2018). As stated in this study, translation assessment belongs to one of the six most disputable but still least numerous topics in the area, being mainly focused on the development and validation of translation tests, grading scales, accreditation, and certification in the translation industry.

The detailed review of the current research state of translation assessment in modern education can be found in many sources (Han, 2021; Hurtado Albir, 2019; Korol, 2019), while the main peculiarities and requirements for the efficient assessment practices in translation training caused by contemporary advances in didactics, foreign language and translation teaching methodologies are summarised in Korol (2018). The latter include the concurrent performance of evaluative and formative functions by the assessment (Klimkowski, 2019; Robinson, Olvera Lobo, & Gutierrez-Artacho, 2013; Wang, 2022), the employment of versatile assessment methods and tools for the measurement of particular assessment objects at its different stages, the evaluation of translation process, product and services as the valuable indicators of the students' actual translation competence level, the arrangement of peer- and self-assessment as intrinsic components of the efficient translation activity in a modern professional environment (Ge & Pi, 2021; Korol, 2020).

Despite the plentitude of both theoretical and practical investigations dedicated to the particular aspects of the abovementioned issues, it is still not always clear in what ways they are reflected and adopted by common practice. According to McAlester (2000), translation assessment procedures vary greatly from country to country, from university to university, from department to department and strongly depend on the teacher's individual experience and preferences. Moreover, they are constantly changing, being flexibly modified and adjusted to the relevant needs and requirements of the language service market. As stated by Huertas Barros & Vine (2018, p. 17), 66.7% out of 27 British universities involved in their studies of the changes in the translation courses for MA training programs in 2015-2016 confirmed significant alterations in their translation assessment practices within the last five years. The main transformations were focused on strengthening of the formative impact of the assessment and the introduction of process-oriented assignments and tasks at different training stages.

The only way to discover and analyses the current state and predict the upcoming trends in the translation assessment practice in a particular country is in the questioning of the parties involved, namely teachers and students, about their personal attitudes and approaches to this process. In particular, the translation teachers may be surveyed from two perspectives: assessment practices in their translation classrooms directed to the studies and improvement of translation pedagogy in general, and assessment of their professional competence outlining personal development needs aimed at the enhancement of translation teachers' education.

The conducted review of available sources on the topic revealed a comparatively limited list of investigations mainly concentrated on specific aspects of assessment practices in the translation training of particular countries. Thus Garant (2009) has carried out a longitude survey on the evaluation practices of Finnish translation teachers in the period of 1997-2008 which indicated the gradual rejection of penalty-based grading and transition to the `assessment for learning' paradigm. Heidari Tabrizi (2021) has analysed the current state of summative assessment in BA translation training programs at Iranian universities and both students' and teachers' attitudes towards it with the help of a survey. As it turned out, all the interviewed cohorts were not satisfied with the quality of summative assessment procedures in translators' education in Iran. Al-Jarf (2021) has studied the structure and content of a summative translation test typical for Saudi Arabian universities. Finally, Li (2006) has conducted one of the most comprehensive surveys. It comprised 95 translation teachers from Chinese universities who were asked about their background, translation training arrangement at their universities, assessment functions and frequency in the translation classroom, assessment tasks and tools developed and applied, and satisfaction with the current state of the assessment practices.

Some general information on the issue can be also obtained from the research projects concerning the desirable features and essential skills of an effective translation teacher (Pavlovic & Antunovic, 2019; Huang & Napier, 2015; Orlando, 2019), including some expectations and dispositions related to the assessment procedures.

Since the situation is constantly changing and developing, such investigations should be conducted periodically in different locations. Therefore, there is some need for studies of current translation assessment practices within particular national contexts, including Ukraine. It should be noted here that similar surveys have never been carried out at Ukrainian universities. In order to fill in the gap and get an insight into the current state of the assessment practices in the domestic translation classroom, we decided to arrange and conduct a survey of university translation teachers in light of contemporary trends in translation teaching and assessment.

Aims and hypothesis. The analysis of the current state of the assessment practices in Ukrainian translation classroom can provide crucial information on their strengths and weaknesses, single out problematic areas and promote the efficiency of the developed and involved assessment procedures. Consequently, the given research aims to accomplish the following:

Results

The completed survey comprised 41 teachers from 16 Ukrainian universities training undergraduate students in Philology (specialism 035 "Philology”, specialisation "Germanic languages and literatures (including translation)”), namely Taras Shevchenko National University, Kyiv National Linguistic University, Kyiv National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv National University of Trade and Economics, National University of Life and Environmental Sciences (Kyiv), National Transport University (Kyiv), Zaporizhzhia National University, Ternopil Volodymyr Hnatiuk National Pedagogical University, Yuriy Fedkovych Chernivtsi National University, Vinnitsia State Pedagogical University, Volodymyr Vynnychenko Central Ukrainian State Pedagogical University, Luhansk Taras Shevchenko National University, National University "Yuri Kondratyuk Poltava Polytechnic”, Poltava State Agrarian University, Poltava V.G. Korolenko National Pedagogical University and Higher Educational Establishment of Ukoopspilka "Poltava University of Economics and Trade”.

The survey participants represent different age categories, inter alia most translation teachers (16 respondents and 39% of the total) are aged from 41 to 50; a bit fewer - 14 (34.2%) - from 31 to 40; 6 (14.6%) - under 30; 4 (9.8%) - from 51 to 60 and only one person (2.4%) - over 61. Females dominated (36 teachers (87.8%)), while there were only 5 males (12.2%) in the analysed sample. Translation teachers' qualifications were distributed as follows: bachelors - 0, specialists - 4 (9.8%), masters - 7 (17.1%), PhDs - 28 (68.3%), Doctors of Sciences - 2 (4.9%). We did not specify in our survey if the respondents held a degree in Translation or Philology. However, we were interested greatly in their practical experience in teaching translation courses at higher educational establishments. At the time of our study, 11 teachers (26.8%) had from one to five and some more 11 surveyed (26.8%) - from 6 to 10 years of such experience. Seven more respondents (17.1%) have been delivering such courses for 11-15 years and for more than 21 years. Finally, the least number of the surveyed, 5 (12.2%), have been teaching translation for 16-20 years. This fact gives the ground to claim the sufficient teachers' proficiency and competence in translation teaching. Demographic data received in the carried-out survey are summarised and presented in Table 1.

Table 1

Respondents' Background

According to the survey results, Ukrainian universities usually do not offer translation practice courses during the first year of studies. However, for some bachelor's programs at four universities involved in our survey (2.5%) it can happen from time to time. This fact can be explained by the permanent changes taking place in flexibly developing Philology curricula. Seven universities (43.75%) out of 16 introduce translation practice courses in the second year of studies. Three more higher educational establishments (18.75%) start teaching translation courses exclusively in the third year of studies. One university (6.25%) provides translation courses to its students either from the second, or third year of studies. In addition, one university (6.25%) offers translation courses only in the fourth year of studies. These differences in curricula structure and content may cause the use of a wide range of assessment practices and inconsistencies in them.

University teachers' attitudes to the importance of the assessment and its functions in translators' training are presented in Table 2.

Table 2

Assessment Importance and Functions in Translation Training

As we can see from Table 2 the majority of the respondents (95.1% in total) find translation assessment important for the efficiency of the training process, only 2 of them consider it neither important, nor unimportant. The leading objectives of translation assessment are claimed to be the definition of the actual proficiency level in terms of students' acquired translation knowledge, developed skills, abilities and competence in general (78.0% of the respondents) and the assessment of the translation course learning outcomes (43.9% of the surveyed teachers) which correspond to its summative and evaluative functions. The promotion of students' learning and translation competence development with the help of the assessment was also highly rated by almost 71% of the teachers. It correlates with its formative function. In addition, about 42% of the respondents acknowledged the importance of the assessment feedback function directed to the evaluation and further correction of the applied teaching methods and tools. Finally, the last question of this section overlaps with the previous one, being aimed at the rating of the importance of the assessment's formative function. The number of teachers who find it important is much higher than those who have chosen it as one of the leading assessment functions responding to the previous question (92.7% versus 70.7%). This fact can be explained by some discrepancies between teachers' theoretical beliefs and real practices. They acknowledge the importance of the assessment's formative impact on the translation training process. However, they do not always apply the appropriate assessment techniques and tools in order to provide the desirable formative effect in the training environment. assessment profile teacher translation university

The next question was devoted to the studies of the assessment frequency in translation training. The teachers were asked about the proper schedule of the students' translation performance monitoring: how often should translation performed by the students be checked up by the teacher? The received results indicated diversified teachers' strategies in terms of this issue. Thus, about a third of the respondents (13 respondents and 31.7%) prefer to assess and correct their students' translation performance on a weekly basis. Two more (4.9%) do it a bit more rarely (about 11-13 times a term). Seven more teachers (29.3%) check up on their students' translations every other week. However, 11 respondents (26.8%) still do it from 4 to 6 times per term which may coincide with the end of a particular unit. Eventually, the other three teachers (7.3%) review their students' translations from 1 to 3 times during the term. In our opinion, translation checkup frequency correlates somehow with the provision of the assessment formative effect. The more often we provide our students with some kind of feedback on their performance, the more productive the training process gets according to the results of the students' survey conducted in 2020 by Korol (Korol, 2021).

The survey results concerning the assessment methods and tools employed by the teachers for the purposes of borderline assessment arrangement in translation training are compiled in Table 3.

According to the obtained data, the majority of teachers (63.4%) use answering theoretical questions in written form as the main tool to assess the acquisition of their students' translation theoretical knowledge. To our mind, this type of task is reminiscent of the conventional examination rather than a borderline test. However, it requires comparatively little prior preparation from the teacher. Multiple choice tests based on answering the questions take the second position in this list with 51.2% of the respondents. Finally, open gap-filling tests are employed by 36.6% of the teachers. The least popular appeared to be oral questioning, explicable by its low practicality in terms of borderline test arrangement. It should be noted here that about 7.0% of the surveyed do not assess their students' translation theoretical knowledge at all within the borderline assessment. As for the assessment of the students' translation practical skills, the most commonly used type of task is still the translation of the whole text or text fragment (61% of the teachers). In addition, the respondents actively applied the tasks to translate separate sentences using the predetermined translation techniques or transformations (53.7%) and the tasks to translate separate sentences (51.2%). It is worth noting that these types of assessment tasks are rather traditional and do not reflect any innovations or recent advances in translation teaching pedagogy. On the other hand, more complicated and progressive tasks have not got sufficient and regular use within borderline assessment in Ukrainian translation classroom.

Table 3

Methods and tools used for borderline assessment arrangement in translation training

For example, the tasks combining theoretical knowledge control and practical performance, namely tasks to translate separate sentences defining the used translation techniques or transformations and tasks to translate the whole text / text fragment defining the used translation techniques or transformations, were employed by 26.8% and 19.5% of the respondents respectively. Both product- and process-oriented tasks such as to translate the whole text / text fragment detecting the main translation problems and suggesting some possible ways to solve them were applied by 34.1% of the surveyed. The tasks containing the elements of self-assessment, including the tasks to translate the whole text / text fragment indicating the simplest and the most difficult passages for rendition and the tasks to translate the whole text / text fragment indicating the most and least successfully rendered passages were used by 24.4% and 34.1% of the teachers. Quite unexpectedly, multiple choice test tasks based on selecting the adequate and equivalent translation for the given sentence/passage, primarily passive by nature, have the support of about 37% of the surveyed translation teachers. As for the increasing trend in assessing students' editing skills due to its importance in the modern language industry, 17.1% of the surveyed translation teachers do not carry out it at all within the borderline assessment. In case of assessing students' editing skills, the respondents mostly used the tasks to detect and correct the errors and mistakes in the given TT (63.4%). On the contrary, the tasks to compare, analyse and evaluate different TT versions for the same ST (56.1%) and the tasks to choose and ground the appropriate TT version from the given options (39.0%) being quite difficult and time-consuming to develop got a rather high rating. The task to evaluate and criticize the given TT was the least extensively used (22.0%) despite its high value for the development of students' peer-assessment skills.

The received data on the assessment methods and tools applied by the teachers for the purposes of summative assessment arrangement in translation training are presented in Table 4. As we can see from Table 4, quite expectedly the number of teachers who do not check up their students' translation theoretical knowledge acquisition has grown from 7.3%, for borderline assessment, up to 12.2% for summative one. The most productive tasks to assess this construct according to 58.5% of the respondents were multiple choice tests based on answering the questions that can be explained by their high practicality in use. Answering the theoretical questions in written form has kept its leading positions here as well, being preferred by 56.1% of the surveyed translation teachers. In our opinion, this fact proves the lack of progressive assessment tools in the modern practice of translation competence assessment. Some interesting trends were observed: the lower percentage of those who apply open-gap filling test tasks to assess the acquisition of translation theoretical knowledge by their students (22.0%) compared to the relevant borderline assessment data and a slight decrease in the use of alternative test tasks (True/False) (19.5% versus 29.3% for borderline assessment). As for the assessment of practical translation skills, the tasks to translate the whole text or text fragment and to translate separate sentences traditionally take the leading positions being employed by about 53% of the respondents.

Table 4

Methods and tools used for summative assessment arrangement in translation training

The other most common task types, in this case, are translating the whole text or text fragment with the definition of the used translation techniques or transformations (that can be treated as a great option in the case of refusal of translation theoretical knowledge assessment within the summative test) and the tasks to translate separate sentences using the pre-determined translation techniques or transformations (36.6% both). Some discrepancies in the arrangement of practical translation skills assessment are fixed. For example, multiple-choice test tasks based on selecting the adequate and equivalent translation for the given sentence/passage are still actively applied by 34.1% of the respondents. Ukrainian translation teachers still underestimate innovative and complicated translation tasks, such as translating the whole text / text fragment accompanied with the indication of the simplest and the most difficult passages for rendition, translating the whole text / text fragment combined with the detection of the main translation problems and suggestion of some possible ways to solve them, and translating the whole text / text fragment with the indication of the most and least successfully rendered passages (24.4%, 7.3% and 24.4% respectively). Finally, almost a quarter of the respondents do not assess their students' editing skills with the help of specifically tailored tasks. Those who still do it prefer quite traditional tasks to detect and correct the errors and mistakes in the given TT (61.0%). In general, translation teachers prefer the same assessment methods and tools for both borderline and summative assessment. It can be treated as some lack of flexibility and realignment in the hierarchy of assessment objects, tools, and assessment types. Arranged in such a way, assessment system loses its formative potential, being focused predominantly on the evaluative function performance and the objectivity of the received grading results.

The next survey questions were devoted to the problem of the indicators of the students' translation competence acquisition, namely the translation product (TT quality) and translation process, taken into account by the teachers in their translation assessment practices. The received results are summarised in Table 5. According to the collected data Ukrainian translation teachers are sure that the assessment of the quality of the performed translation serves quite an objective indicator of their students' translation competence level (78.1% of the respondents either strongly agree with or just support this idea). However, 22.0% of the respondents have some doubt as to whether the received target text quality can provide sufficient information on this matter. At the same time, even more of the surveyed (85.4% altogether) believe that translation process monitoring can serve as a source of reliable and valid information on their students' translation competence level. Nevertheless, 12.2% of the respondents are quite hesitant about it and even one teacher disagrees with this idea. The survey results discussed above prove that in spite of the fact that the majority of the teachers are quite aware of the need for translation process monitoring, they do not put these beliefs into their teaching practice, since relevant assessment tasks are not widely used in the training process. The next step of our survey was to find out what kind of alternative assessment tools are utilized by Ukrainian translation teachers. The obtained results are given in Table 6 and discussed below.

Table 5

Translation teachers' attitudes to translation product and process as the indicators of the students' translation competence

Table 6

Alternative assessment tools in translation training

*the comments given in parenthesis were not present in the online questionnaire and were added here for the sake of a more explicit interpretation of the received data

As we can see from Table 6, about 20% of the respondents do not apply any alternative assessment tools in teaching translation to their students at all. The most popular assessment tool preferred by nearly 50% of the surveyed translation teachers is the translation project, which can be implemented differently in the teaching process. It can be both process- and product-oriented, provide conditions for the assessment performance by different agents (self-, peer-, hetero- and group assessment) one at a time or in various combinations. Translation portfolio appeared to be the second most common alternative assessment tool used by almost 30% of the survey participants. The other types of alternative assessment tools were scattered among an insignificant number of respondents, although they were quite open to new opportunities for translation process monitoring, self- and peer- assessment arrangement.

That is why the next aspect we were interested in was the translation teachers' attitude to the peer-assessment role in the teaching process. The obtained data are collected in Table 7.

Table 7

Peer-assessment arrangement in translation training

According to the received data, almost 76% of the respondents considered peer- assessment important for translation training. On the contrary, 22% of them were not sure about its positive impact on the training outcomes. At the same time, just about 20% of the surveyed teachers regularly developed and supplied their students with the tools to arrange peer-assessment of their translation performance; the other 37% did it from time to time. The remaining 39% only motivated and urged their students to practise peer assessment. The development of the students' peer assessment skills should serve as the basis for the efficient introduction of self-assessment practices in the translation classroom. Teachers' opinion concerning self-assessment in translators' training is presented in Table 8.

Table 8

Self-assessment arrangement in translation training

According to the received data, almost 83% of the respondents highly rated the importance of self-assessment in translation training, while only 17% of them were hesitant about its positive impact on the training outcomes. However, only 4.9% of the teachers regularly provided their students with relevant tools and 46.3% more did it sporadically. The remaining 43.9% just motivated and urged their students to self-assess their translation performance, which can hardly be treated as an efficient training strategy.

The next issue we inquired about was the assessment methods used by Ukrainian translation teachers to evaluate their students' works. The obtained information is summarized in Table 9.

Table 9

Translation assessment methods

According to most of the respondents (about 73%), translation assessment results should be delivered to students in the form of a quantitative grade and some kind of qualitative feedback (verbal, graphical or not) at a time. However, almost 12% of the teachers still believed that a conventional grade would be enough. These data somehow correlate with the respondents' opinion about the importance of teacher feedback on students' translation for the development of their practical translation skills (97.6% of the surveyed find it either extremely or just important).

As for the grading methods employed by the teachers for students' translation evaluation about 30% of the translation teachers preferred analytical grading based on predetermined criteria to assess their students' translation product, which can be rather functional for the delivery of verbal feedback. Almost 27% of them used a holistic grading method based on their general impression of the TT, which seems to be less objective and less productive in terms of verbal feedback. A positive trend is seen in the fact that nearly 17% of the respondents applied grading according to the successful translation solutions. It is the opposite to the least common penalty-based grading method, which requires error and mistake detection in the students' TT and deduction of the relevant points out of 100% according to the severity or weight of the fixed mistakes. Nevertheless, 12.2% of the translation teachers practised it. Being rather informative from the viewpoint of detailed feedback delivery (Amini, 2018), this grading method was condemned for its demotivational and destructive impact on students' translation performance since it neglects productive translation solutions and does not provide any space for improvement and development. Almost 88% of the respondents insisted on the objectivity of the assessment results obtained with the help of the applied grading methods commented on above. Only 12.2% of the teachers expressed hesitation on the objectivity of their assessment practices.

The next issues we wanted to discover were the main problems Ukrainian translation teachers face in terms of assessment arrangement and the factors that cause them. The received responses were processed and compiled into Table 10.

Table 10

Typical problems of translation assessment arrangement in Ukrainian classroom

From Table 10 we can conclude that self-assessment arrangement and realization appear to be the most challenging for Ukrainian translation teachers (about half of the surveyed). These data conform to the fact that about 83% of the respondents are quite aware of its importance for the efficiency of translators' training; however, only half of the translation teachers implement some measures to arrange it. Despite the high estimation of the objectivity of the applied grading methods, about 40% of the respondents find this issue quite problematic. The selection of the text materials for translation tasks quite unexpectedly took the third leading position in this list (almost 32% of the survey participants). To our mind, that is one of the primary problems faced by the teachers who take their first steps in translation teaching. On the other hand, this problem may arise again if it takes into account the great number of different text types and domains with their specific peculiarities to be analysed and represented in line with the particular translation training curriculum. It is interesting that feedback assessment function got the lowest response rate among the surveyed teachers and only 26.8% of them accepted some difficulties connected with the response on the assessment results. Despite insufficient level of peer-assessment arrangement in the translation classroom (see Table 7), only a quarter of the respondents were worried about this translation assessment aspect. One more discrepancy is observed in terms of the diversity and number of the existing assessment tools (less than 10% of the surveyed felt some lack of them and about 22% of them had problems with their selection) and quite a limited list of them were used by the respondents in their assessment practice (see Tables 3, 4 and 6). As for the factors causing the difficulties commented on above, the lack of time to check up and grade students' translation...