Journal Article

Exploring the Efficacy of ChatGPT-Based Feedback Compared With Teacher Feedback and Self-Feedback: Evidence From Chinese-English Translation

Zhou, Tongquan, Cao, Siyi

2025

Overview

ChatGPT, a cutting-edge AI-powered Chatbot, can quickly generate responses to given commands. While ChatGPT was reported to have the capacity to deliver useful feedback, it is still unclear about its effectiveness compared with conventional feedback approaches, such as self-feedback (SF) and teacher feedback (TF). To address this issue, this study compared the revised Chinese to English translation texts produced by 45 Chinese Master of Translation and Interpretation (MTI) students, who learned English as a Second Language (ESL), based on three feedback types (i.e., SF, TF, and ChatGPT feedback). The data was analyzed using BLEU score to gauge the overall translation quality as well as Coh-Metrix to examine linguistic features across three dimensions: lexicon, syntax, and cohesion. The findings revealed that SF and TF-guided translation texts surpassed those with ChatGPT feedback, as indicated by the BLEU score. In terms of linguistic features, ChatGPT feedback demonstrated superiority, particularly in enhancing lexical capability and referential cohesion in the translation texts. However, SF and TF proved more effective in developing syntax-related skills, as they addressed instances of incorrect usage of the passive voice. These diverse outcomes indicate ChatGPT’s potential as a supplementary resource, complementing traditional teacher-led methods in translation practice. Plain language summary Assessing the Efficacy of ChatGPT-based Feedback in Chinese to English Translation: A Comparative Study with Teacher and Self-Feedback Feedback plays a crucial role in the process of learning English as a second language (ESL), as it supports student motivation and achievement. ChatGPT, a cutting-edge AI-powered Chatbot, can aid ESL learners by providing instant and personalized feedback proved by theoretical studies. However, it is still unclear about its effectiveness compared with conventional feedback approaches, such as teacher feedback (TF) and self-feedback (SF). The aim of the present study is to compare the quality of Chinese to English translation texts produced by Chinese Master of Translation and Interpretation (MTI) students based on three feedback types (i.e., ChatGPT-based feedback, TF, and SF). A total of 135 translation texts were collected from 45 MTI participants, each subjected to three rounds of feedback-driven revisions. Our analysis framework encompassed two main aspects: the overall translation quality and the linguistic dimensions. The findings contribute to the ongoing discussion about the role of AI by highlighting the specific strengths and weaknesses of ChatGPT in translator training. From a theoretical standpoint, the findings illuminate the limitations of AI in handling the complexities of human linguistic abilities, particularly in the realm of syntax, suggesting a need for further development in this area. On a practical level, our study indicates ChatGPT’s potential as a supplementary resource, complementing traditional teacher-led methods in translation practice.

• Publisher: SAGE Publications,SAGE PUBLICATIONS, INC,SAGE Publishing
• Subject: Academic achievement / Artificial intelligence / Chatbots / Chinese languages / Cohesion / Comparative analysis / Comparative linguistics / Comparative studies / Computer assisted language learning / Data quality / Efficacy / English / English (Second Language) / English as a second language / English as a second language instruction / English language / English Learners / Feedback / Grammar lexicon relationship / Human-computer interaction / Imperative sentences / Learning / Linguistics / Motivation / Second language teachers / Second Languages / Student Motivation / Students / Syntax / Teachers / Teaching methods / Translation / Translation instruction / Translation methods and strategies / Translations / Translators