Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
9-2020
Abstract
Neural networks are becoming a popular tool for solving many realworld problems such as object recognition and machine translation, thanks to its exceptional performance as an end-to-end solution. However, neural networks are complex black-box models, which hinders humans from interpreting and consequently trusting them in making critical decisions. Towards interpreting neural networks, several approaches have been proposed to extract simple deterministic models from neural networks. The results are not encouraging (e.g., low accuracy and limited scalability), fundamentally due to the limited expressiveness of such simple models.In this work, we propose an approach to extract probabilistic automata for interpreting an important class of neural networks, i.e., recurrent neural networks. Our work distinguishes itself from existing approaches in two important ways. One is that probability is used to compensate for the loss of expressiveness. This is inspired by the observation that human reasoning is often 'probabilistic'. The other is that we adaptively identify the right level of abstraction so that a simple model is extracted in a request-specific way. We conduct experiments on several real-world datasets using state-of-the-art architectures including GRU and LSTM. The result shows that our approach significantly improves existing approaches in terms of accuracy or scalability. Lastly, we demonstrate the usefulness of the extracted models through detecting adversarial texts.
Keywords
Abstraction, Interpretation, Probabilistic automata, Recurrent neural networks
Discipline
Software Engineering
Research Areas
Software and Cyber-Physical Systems
Publication
2020 35th IEEE/ACM International Conference on Automated Software Engineering ASE: Virtual, September 21-25: Proceedings
First Page
499
Last Page
510
ISBN
9781450367684
Identifier
10.1145/3324884.3416592
Publisher
ACM
City or Country
New York
Embargo Period
5-17-2021
Citation
DONG, Guoliang; WANG, Jingyi; SUN, Jun; ZHANG, Yang; WANG, Xinyu; DAI, Ting; DONG, Jin Song; and WANG, Xingen.
Towards interpreting recurrent neural networks through probabilistic abstraction. (2020). 2020 35th IEEE/ACM International Conference on Automated Software Engineering ASE: Virtual, September 21-25: Proceedings. 499-510.
Available at: https://ink.library.smu.edu.sg/sis_research/5947
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3324884.3416592