doi: 10.18178/ijiet.2026.16.1.2479
A Multi-Criteria Software Quality Evaluation of AI Meeting Assistants for English-Medium University Lectures Using ISO/IEC 25010 and TOPSIS
Alex Pak Ki Kwok1,*
,
Yao Hing Wong2 ,
Kwong-Cheong Wong1 ,
Chee Hon Chan1
- 1Data Science and Policy Studies Programme, School of Governance and Policy Science, Faculty of Social Science, The Chinese University of Hong Kong, Hong Kong, China
- 2English Language Teaching Unit, The Chinese University of Hong Kong, Hong Kong, China
- Manuscript receivedMay 28, 2025
- revisedJune 20, 2025
- acceptedJuly 28, 2025
Abstract
This study aims to systematically evaluate the effectiveness of ten Artificial Intelligence Meeting Assistants (AIMAs) in supporting English-medium university lectures. The research was carried out at The Chinese University of Hong Kong and involved both teachers and students as stakeholders. Using a within-subjects design, twenty participants (twelve students and eight teaching staff, recruited through snowball sampling across multiple faculties) tested each AIMA in simulated lecture contexts. Data were collected through structured questionnaires based on the ISO/IEC 25010 software quality framework, covering nine criteria including functional suitability, performance efficiency, compatibility, usability, reliability, security, satisfaction, sustainability, and scalability. The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) was used to combine participants’ importance ratings and performance scores, resulting in a final ranking of the AIMAs. TOPSIS analysis of participant evaluations ranked Tl;dv most favorably, followed by Grain and Microsoft Teams. Notably, teachers rated security (p < 0.001) and performance efficiency (p = 0.009) significantly higher than students, highlighting differing user priorities. This study provides empirical benchmarks and a replicable framework for selecting educational technologies. The findings may help institutions make evidence-based decisions about using AIMAs to improve student understanding and participation in linguistically diverse classrooms.
Keywords
- Artificial Intelligence (AI) meeting assistants
- educational technology
- English-medium instruction
- ISO/IEC 25010
- Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS)
How to Cite
Alex Pak Ki Kwok, Yao Hing Wong, Kwong-Cheong Wong, and Chee Hon Chan, "A Multi-Criteria Software Quality Evaluation of AI Meeting Assistants for English-Medium University Lectures Using ISO/IEC 25010 and TOPSIS," International Journal of Information and Education Technology, vol. 16, no. 1, pp. 16-26, 2026. https://doi.org/10.18178/ijiet.2026.16.1.2479
Copyright & License
Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).