Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
8-2019
Abstract
This paper presents a simple yet principled approach to boosting the robustness of the residual network (ResNet) that is motivated by a dynamical systems perspective. Namely, a deep neural network can be interpreted using a partial differential equation, which naturally inspires us to characterize ResNet based on an explicit Euler method. This consequently allows us to exploit the step factor h in the Euler method to control the robustness of ResNet in both its training and generalization. In particular, we prove that a small step factor h can benefit its training and generalization robustness during backpropagation and forward propagation, respectively. Empirical evaluation on real-world datasets corroborates our analytical findings that a small h can indeed improve both its training and generalization robustness.
Keywords
Machine Learning, Deep Learning
Discipline
Artificial Intelligence and Robotics
Research Areas
Intelligent Systems and Optimization
Areas of Excellence
Digital transformation
Publication
Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19), Macao, China, August 10-16
First Page
4285
Last Page
4291
ISBN
9780999241141
Identifier
10.24963/ijcai.2019/595
Publisher
IJCAI
City or Country
Macao
Citation
ZHANG, Jingfeng; HAN, Bo; WYNTER, Laura; LOW, Bryan Kian Hsiang; and KANKANHALLI, Mohan.
Towards robust resNet: A small step but a giant leap. (2019). Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19), Macao, China, August 10-16. 4285-4291.
Available at: https://ink.library.smu.edu.sg/sis_research/10322
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.24963/ijcai.2019/595