International Journal of
Information and Education Technology

Abstract

Generative Artificial Intelligence (GAI) like Chat Generative Pre-Trained Transformer (ChatGPT) has been widely used by English as a Foreign Language (EFL) students to polish their academic writing, but little quantitative research has explored how the chatbot can revise human writing. Addressing this research gap is crucial, as understanding the potential of GAI in enhancing the quality of EFL writings can inform both pedagogical practices and the development of computer-assisted writing tools. In response to the issue, this study compared five textual features of Chinese EFL learners’ academic writings and their revised versions of ChatGPT through a computational analysis tool, Coh-Metrix. The results of the Wilcoxon Signed-Rank Test showed that the GAI could significantly enhance the writing quality of handwritten texts only in narrativity and deep cohesion. By illustrating the performance of ChatGPT as an editor, this study has provided insight into pedagogical instruction on English academic writing and computer-assisted writing.