Publication Type

Conference Proceeding Article

Version

acceptedVersion

Publication Date

12-2026

Abstract

Ensuring the quality of multiple-choice questions (MCQs) remains a complex challenge in higher education, as traditional post-hoc psychometric analyses often identify flaws only after students have been negatively impacted. While Large Language Models (LLMs) offer potential for simulating student performance to validate items before delivery, they are frequently hindered by a “high-accuracy bias” that prevents larger models from realistically mimicking student errors. This paper proposes the Triadic Pedagogical Fit (TPF) framework, which triangulates Model Scale, Task Complexity via Revised Bloom’s Taxonomy, and Instructional Persona to optimize pre-assessment evaluation. Through two studies, we demonstrate that while Mid-scale (12B) models effectively mirror undergraduate performance via high task sensitivity, Large-scale (27B) models require a “Teacher Persona” to bypass their high-accuracy bias. This pedagogical positioning transforms advanced models into superior “Item Auditors” for identifying ambiguous questions. We conclude with a decision matrix to guide instructors in selecting the optimal LLM configuration for assessment evaluation.

Keywords

Pre-assessment evaluation, Student performance stimulation, Large Language Model

Discipline

Artificial Intelligence and Robotics | Education

Research Areas

Intelligent Systems and Optimization

Areas of Excellence

Digital transformation

Publication

Proceedings of the Forty-Seventh International Conference on Information Systems (ICIS 2026), Lisbon, Portugal, December 13-16

First Page

1

Last Page

9

Publisher

AIS

City or Country

Lisbon

Download

Share

COinS