Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
12-2020
Abstract
We uncover an ever-overlooked deficiency in the prevailing Few-Shot Learning (FSL) methods: the pre-trained knowledge is indeed a confounder that limits the performance. This finding is rooted from our causal assumption: a Structural Causal Model (SCM) for the causalities among the pre-trained knowledge, sample features, and labels. Thanks to it, we propose a novel FSL paradigm: Interventional Few-Shot Learning (IFSL). Specifically, we develop three effective IFSL algorithmic implementations based on the backdoor adjustment, which is essentially a causal intervention towards the SCM of many-shot learning: the upper-bound of FSL in a causal view. It is worth noting that the contribution of IFSL is orthogonal to existing fine-tuning and meta-learning based FSL methods, hence IFSL can improve all of them, achieving a new 1-/5-shot state-of-the-art on miniImageNet, tieredImageNet, and cross-domain CUB. Code is released at https://github.com/yue-zhongqi/ifsl.
Keywords
Causal intervention, Causal model, Cross-domain, Fine tuning, Interventional, Metalearning, Sample features, State of the art
Discipline
Artificial Intelligence and Robotics | Databases and Information Systems
Research Areas
Intelligent Systems and Optimization
Publication
Advances in Neural Information Processing Systems: Proceedings of the 34th Conference NeurIPS 2020, Vancouver, Canada, December 6-12, Virtual
First Page
1
Last Page
23
Publisher
MIT
City or Country
Cambridge, MA
Citation
YUE, Zhongqi; ZHANG Hanwang; SUN, Qianru; and HUA, Xian-Sheng.
Interventional few-shot learning. (2020). Advances in Neural Information Processing Systems: Proceedings of the 34th Conference NeurIPS 2020, Vancouver, Canada, December 6-12, Virtual. 1-23.
Available at: https://ink.library.smu.edu.sg/sis_research/5596
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://proceedings.neurips.cc/paper/2020/file/1cc8a8ea51cd0adddf5dab504a285915-Paper.pdf