IJIET 2026 Vol.16(3): 695-707
doi: 10.18178/ijiet.2026.16.3.2542
doi: 10.18178/ijiet.2026.16.3.2542
Clinical Reasoning-Driven Progress Evaluation of Medical Students Using Large Language Models
Heitor S. Mattosinho1,*, Fernando Valente2, Gabriel Leite1, Ligia Maria Cayres Ribeiro2,
Marco A. de Carvalho Filho2, and
André Santanchè1
1. Institute of Computing, University of Campinas, Brazil
2. University Medical Center Groningen, University of Groningen, Netherlands
Email: heitor.mattosinho@ic.unicamp.br (H.S.M.); f.valente@hc.fm.usp.br (F.V.); gabriel.dfleite@gmail.com (G.L.); l.m.cayres.ribeiro@umcg.nl (L.M.C.R.); m.a.de.carvalho.filho@umcg.nl (M.A.C.F.); santanche@ic.unicamp.br (A.S.)
*Corresponding author
2. University Medical Center Groningen, University of Groningen, Netherlands
Email: heitor.mattosinho@ic.unicamp.br (H.S.M.); f.valente@hc.fm.usp.br (F.V.); gabriel.dfleite@gmail.com (G.L.); l.m.cayres.ribeiro@umcg.nl (L.M.C.R.); m.a.de.carvalho.filho@umcg.nl (M.A.C.F.); santanche@ic.unicamp.br (A.S.)
*Corresponding author
Manuscript received August 14, 2025; revised September 8, 2025; accepted November 6, 2025; published March 13, 2026
Abstract—Evaluating medical students’ written answers to questions on a given topic can provide information about their mental representations of a disease―i.e., illness script. However, limitations in methods for assessing how medical students develop clinical expertise hinder the advancement of educational practices. This study, therefore, proposes a technique that utilizes semantic annotations of the students’ answers to trace a map of their knowledge concerning the topic of a question. Since manual text annotation is time- and effort-intensive, this study developed an innovative, illness-script-driven strategy using large language models. It identifies relevant medical information in the texts, creates a profile for each answer, clusters them, and categorizes the clusters. Practical experiments with Brazilian students demonstrate that the technique automatically traces consistent profiles from the responses, quantifying how knowledge of disease diagnosis evolves throughout the medical course.
Keywords—illness script, medical education evaluation, large language model, clustering profiles
Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
Keywords—illness script, medical education evaluation, large language model, clustering profiles
Cite: Heitor S. Mattosinho, Fernando Valente, Gabriel Leite, Ligia Maria Cayres Ribeiro, Marco A. de Carvalho Filho, and André Santanchè, "Clinical Reasoning-Driven Progress Evaluation of Medical Students Using Large Language Models," International Journal of Information and Education Technology, vol. 16, no. 3, pp. 695-707, 2026.
Copyright © 2026 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).