Publication Type
Conference Proceeding Article
Book Title/Conference/Journal
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019 July 28 - August 2
Year
7-2019
Abstract
This paper tackles the problem of reading comprehension over long narratives where documents easily span over thousands of tokens. We propose a curriculum learning (CL) based Pointer-Generator framework for reading/sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty. This can be interpreted as a form of domain randomization and/or generative pretraining during training. To this end, the usage of the Pointer-Generator softens the requirement of having the answer within the context, enabling us to construct diverse training samples for learning. Additionally, we propose a new Introspective Alignment Layer (IAL), which reasons over decomposed alignments using block-based self-attention. We evaluate our proposed method on the NarrativeQA reading comprehension benchmark, achieving state-of-the-art performance, improving existing baselines by 51% relative improvement on BLEU-4 and 17% relative improvement on Rouge-L. Extensive ablations confirm the effectiveness of our proposed IAL and CL components.
Disciplines
OS and Networks
Subject(s)
Applied or Integration/Application Scholarship
Publisher
ACL
DOI
10.18653/v1/P19-1486
Version
publishedVersion
Language
eng
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Format
application/PDF
Citation
TAY, Yi; WANG, Shuohang; LUU, Anh Tuan; FU, Jie; PHAN, Minh C.; YUAN, Xingdi; RAO, Jinfeng; HUI, Siu Cheung; and ZHANG, Aston.
Simple and effective curriculum pointer-generator networks for reading comprehension over long narratives. (2019). Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019 July 28 - August 2. 4922-4931.
Available at: https://ink.library.smu.edu.sg/scis_studentpub/1
Additional URL
https://doi.org/10.18653/v1/P19-1486
