Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

6-2025

Abstract

Identifying logical fallacies is essential for maintaining log-ical reasoning and reducing false information in a variety of domains, such as the media, law, and education. We present an extensive study on the use of large language models (LLMs) for logical fallacy detection and provide a comparative overview of model performance across various fallacy classes. We evaluate the logical fallacy detection capabilities of multiple state-of-the-art models (LLaMA, Qwen, Gemma, Phi) utilizing accuracy, precision, recall, and F1-score as assessment measures. Accord-ing to our findings, our models do well on simple fallacies like “circular reasoning,” but they have trouble with more interpretive reasoning when it comes to more complex categories like “equivocation” and “intentional”. These results highlight the potential of LLMs in fallacy detection tasks but also indicate a need for improved prompt engineering, fine-tuning, and context-rich datasets to enhance interpretive accuracy. This research offers insights into advancing LLMs for critical reasoning applications, contributing to improved information integrity across domains.

Keywords

Large language models, Logical fallacies, Reasoning

Discipline

Databases and Information Systems | Programming Languages and Compilers

Research Areas

Data Science and Engineering

Areas of Excellence

Digital transformation

Publication

PAKDD 2025: Workshops, ADUR FairPC, GLFM, PM4B and RAFDA Sydney, June 10-13, Proceedings

First Page

387

Last Page

398

ISBN

9789819681969

Identifier

10.1007/978-981-96-8197-6_29

Publisher

Springer

City or Country

Cham

Additional URL

https://doi.org/10.1007/978-981-96-8197-6_29

Download

Share

COinS