Publication Type

Journal Article

Version

acceptedVersion

Publication Date

4-2022

Abstract

Transfer learning plays an essential role in Deep Learning, which can remarkably improve the performance of the target domain, whose training data is not sufficient. Our work explores beyond the common practice of transfer learning with a single pre-trained model. We focus on the task of Vietnamese sentiment classification and propose LIFA, a framework to learn a unified embedding from several pre-trained models. We further propose two more LIFA variants that encourage the pre-trained models to either cooperate or compete with one another. Studying these variants sheds light on the success of LIFA by showing that sharing knowledge among the models is more beneficial for transfer learning. Moreover, we construct the AISIA-VN-Review-F dataset, the first large-scale Vietnamese sentiment classification database. We conduct extensive experiments on the AISIA-VN-Review-F and existing benchmarks to demonstrate the efficacy of LIFA compared to other techniques. To contribute to the Vietnamese NLP research, we publish our source code and datasets to the research community upon acceptance.

Keywords

LIFA, Low-resource NLP, Mixture of experts, Sentiment classification, Transfer learning

Discipline

Databases and Information Systems | East Asian Languages and Societies

Publication

Information Sciences

Volume

590

First Page

1

Last Page

14

ISSN

0020-0255

Identifier

10.1016/j.ins.2021.12.059

Publisher

Elsevier

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1016/j.ins.2021.12.059

Download

Share

COinS