Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
9-2021
Abstract
Understanding idioms is important in NLP. In this paper, we study to what extent pre-trained BERT model can encode the meaning of a potentially idiomatic expression (PIE) in a certain context. We make use of a few existing datasets and perform two probing tasks: PIE usage classification and idiom paraphrase identification. Our experiment results suggest that BERT indeed can separate the literal and idiomatic usages of a PIE with high accuracy. It is also able to encode the idiomatic meaning of a PIE to some extent.
Discipline
Databases and Information Systems | Programming Languages and Compilers
Research Areas
Data Science and Engineering
Publication
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), Virtual Conference, September 1-3
First Page
1397
Last Page
1407
Identifier
10.26615/978-954-452-072-4_156
Publisher
Incoma Ltd.
City or Country
Virtual Conference
Citation
TAN, Minghuan and JIANG, Jing.
Does BERT understand idioms? A probing-based empirical study of BERT encodings of idioms. (2021). Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), Virtual Conference, September 1-3. 1397-1407.
Available at: https://ink.library.smu.edu.sg/sis_research/6722
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.