doi: 10.18178/ijiet.2026.16.3.2542
Clinical Reasoning-Driven Progress Evaluation of Medical Students Using Large Language Models
Heitor S. Mattosinho1,*
,
Fernando Valente2 ,
Gabriel Leite1 ,
Ligia Maria Cayres Ribeiro2 ,
Marco A. de Carvalho Filho2 ,
André Santanchè1
- 1Institute of Computing, University of Campinas, Brazil
- 2University Medical Center Groningen, University of Groningen, Netherlands
- Manuscript receivedAugust 14, 2025
- revisedSeptember 8, 2025
- acceptedNovember 6, 2025
- publishedMarch 13, 2026
Abstract
Evaluating medical students’ written answers to questions on a given topic can provide information about their mental representations of a disease―i.e., illness script. However, limitations in methods for assessing how medical students develop clinical expertise hinder the advancement of educational practices. This study, therefore, proposes a technique that utilizes semantic annotations of the students’ answers to trace a map of their knowledge concerning the topic of a question. Since manual text annotation is time- and effort-intensive, this study developed an innovative, illness-script-driven strategy using large language models. It identifies relevant medical information in the texts, creates a profile for each answer, clusters them, and categorizes the clusters. Practical experiments with Brazilian students demonstrate that the technique automatically traces consistent profiles from the responses, quantifying how knowledge of disease diagnosis evolves throughout the medical course.
Keywords
- illness script
- medical education evaluation
- large language model
- clustering profiles
How to Cite
Heitor S. Mattosinho, Fernando Valente, Gabriel Leite, Ligia Maria Cayres Ribeiro, Marco A. de Carvalho Filho, and André Santanchè, "Clinical Reasoning-Driven Progress Evaluation of Medical Students Using Large Language Models," International Journal of Information and Education Technology, vol. 16, no. 3, pp. 695-707, 2026. https://doi.org/10.18178/ijiet.2026.16.3.2542
Copyright & License
Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).