Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

4-2022

Abstract

Hateful meme detection is a new multimodal task that has gained significant traction in academic and industry research communities. Recently, researchers have applied pre-trained visual-linguistic models to perform the multimodal classification task, and some of these solutions have yielded promising results. However, what these visual-linguistic models learn for the hateful meme classification task remains unclear. For instance, it is unclear if these models are able to capture the derogatory or slurs references in multimodality (i.e., image and text) of the hateful memes. To fill this research gap, this paper propose three research questions to improve our understanding of these visual-linguistic models performing the hateful meme classification task. We found that the image modality contributes more to the hateful meme classification task, and the visual-linguistic models are able to perform visual-text slurs grounding to a certain extent. Our error analysis also shows that the visual-linguistic models have acquired biases, which resulted in false-positive predictions.

Keywords

Explainable machine learning, Hate speech, Hateful memes, Multimodal

Discipline

Databases and Information Systems

Research Areas

Data Science and Engineering

Publication

Proceedings of the 31st ACM World Wide Web Conference, Virtual, Online, 2022 April 25-29

First Page

3651

Last Page

3655

ISBN

9781450390965

Identifier

10.1145/3485447.3512260

Publisher

ACM

City or Country

New York

Additional URL

https://doi.org/10.1145/3485447.3512260

Download

Share

COinS