IJIET 2026 Vol.16(8): 2107-2116
doi: 10.18178/ijiet.2026.16.8.2670
doi: 10.18178/ijiet.2026.16.8.2670
A Quasi-experimental Study of Conversational AI-supported Engagement in Large CS Courses
Neha Rani 1,*, Sharan Majumder 1, Ishan Bhardwaj 1, and Pedro Guillermo Feijóo-García 2
1. Department of Computer and Information Science and Engineering, University of Florida, Gainesville, USA
2. School of Computing Instruction, College of Computing, Georgia Institute of Technology, Atlanta, USA
Email: neharani@ufl.edu (N.R.); smajumder1@ufl.edu (S.M.); ishanbhardwaj@ufl.edu (I.B.); pfeijoogarcia@gatech.edu (P.G.F.-G.)
*Corresponding author
2. School of Computing Instruction, College of Computing, Georgia Institute of Technology, Atlanta, USA
Email: neharani@ufl.edu (N.R.); smajumder1@ufl.edu (S.M.); ishanbhardwaj@ufl.edu (I.B.); pfeijoogarcia@gatech.edu (P.G.F.-G.)
*Corresponding author
Manuscript received October 13, 2025; revised December 3, 2025; accepted April 20, 2026; published August 18, 2026
Abstract—Lucrative career prospects and creative opportunities often attract students to enroll in computer science courses, resulting in large class sizes. A common challenge in large classrooms is the lack of engagement between students and both the instructor and the learning material. Recent advances in Large Language Models (LLMs) present an opportunity to leverage Conversational Artificial Intelligence (CAI) to address this challenge. To examine the potential of CAI to support engagement with learning material, we designed an in-class activity in a large Software Engineering course in which students interacted with a CAI tool. Using a quasi-experimental, within-subject study design in a real classroom at a large public United States University. We compared student engagement during an activity that involved the use of CAI for discussion with an activity that did not (peer discussion). For our analysis, we collected data on students’ interactions with the CAI tool and their self-reported responses on multiple engagement scales. Results show higher student engagement with the learning material in the activity involving CAI discussion as opposed to without (peer discussion). Higher self-reported learning was also observed in the activity with CAI as compared to without. Students also reported significantly more usefulness for learning in the CAI condition. Overall, our results indicate that CAI (ChatGPT) has the potential to support engagement with learning content during in-class activities in large class sizes.
Keywords—Conversational Artificial Intelligence (CAI), ChatGPT, large classes, engagement, learning outcome
Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
Keywords—Conversational Artificial Intelligence (CAI), ChatGPT, large classes, engagement, learning outcome
Cite: Neha Rani, Sharan Majumder, Ishan Bhardwaj, and Pedro Guillermo Feijóo-García, "A Quasi-experimental Study of Conversational AI-supported Engagement in Large CS Courses," International Journal of Information and Education Technology, vol. 16, no. 8, pp. 2107-2116, 2026.
Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).