Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

5-2026

Abstract

Reflective learning enhances understanding, especially when instructors promptly address difficulties raised in student reflections. Automated doubt detection can reduce time for instructors, yet existing classification approaches take substantial time for manual annotation and model training. This paper investigates whether large and small language models (LLMs, SLMs) can automate doubt detection without time-consuming training. Using a dataset of anonymized student reflections, we evaluate zeroshot, few-shot prompting, and multi-step reasoning against prior supervised classification baselines. We show that LLMs (GPT-4o, Claude-4, Gemini-2.5) surpass earlier F1 scores without prompting, while prompting further improves their performance. However, using proprietary LLMs can raise cost and privacy concerns. We also show that selected SLMs (Mistral, Qwen) outperform baselines while addressing these concerns. We extend the analysis with a category-level error study, showing that explicit doubts are detected more reliably, while tentative doubts (softened by cautious language), learning challenge (arising from difficulties in applying concepts), and masked doubts (concealed by positive or polite phrasing) are missed more often. These findings highlight both the promise and limitations of language models for doubt detection and the need to ensure that cautious or polite learners who may not express their doubts explicitly are recognized and supported in learning analytics systems.

Keywords

Doubt Identification, Reflective Learning, LLM, Generative AI, Prompt Design and Engineering

Discipline

Artificial Intelligence and Robotics

Research Areas

Intelligent Systems and Optimization

Areas of Excellence

Digital transformation

Publication

LAK '26: Proceedings of the LAK26: 16th International Learning Analytics and Knowledge Conference, Bergen, Norway, April 27 - May 1

First Page

117

Last Page

126

ISBN

9798400720666

Identifier

10.1145/3785022.3785037

Publisher

ACM

City or Country

New York

Additional URL

https://doi.org/10.1145/3785022.3785037

Download

Share

COinS