Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
7-2020
Abstract
In this paper, we study Multimodal Named Entity Recognition (MNER) for social media posts. Existing approaches for MNER mainly suffer from two drawbacks: (1) despite generating word-aware visual representations, their word representations are insensitive to the visual context; (2) most of them ignore the bias brought by the visual context. To tackle the first issue, we propose a multimodal interaction module to obtain both image-aware word representations and word-aware visual representations. To alleviate the visual bias, we further propose to leverage purely text-based entity span detection as an auxiliary module, and design a Unified Multimodal Transformer to guide the final predictions with the entity span predictions. Experiments show that our unified approach achieves the new state-of-the-art performance on two benchmark datasets.
Discipline
Databases and Information Systems | Theory and Algorithms
Research Areas
Data Science and Engineering
Publication
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
First Page
3342
Last Page
3352
Identifier
10.18653/v1/2020.acl-main.306
Publisher
Association for Computational Linguistics
City or Country
Online
Citation
YU, Jianfei; Jing JIANG; YANG, Li; and XIA, Rui.
Improving multimodal named entity recognition via entity span detection with unified multimodal transformer. (2020). Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 3342-3352.
Available at: https://ink.library.smu.edu.sg/sis_research/5272
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.18653/v1/2020.acl-main.306