Text backdoor detection using an interpretable RNN abstract model
Publication Type
Journal Article
Publication Date
1-2021
Abstract
Deep neural networks (DNNs) are known to be inherently vulnerable to malicious attacks such as the adversarial attack and the backdoor attack. The former is crafted by adding small perturbations to benign inputs so as to fool a DNN. The latter generally embeds a hidden pattern in a DNN by poisoning the dataset during the training process, which causes the infected model to misbehave on predefined inputs with a specific trigger and normally perform for others. Much work has been conducted on defending against the adversarial samples, while the backdoor attack received much less attention, especially in recurrent neural networks (RNNs), which play an important role in the text processing field. Two main limitations make it hard to directly apply existing image backdoor detection approaches to RNN-based text classification systems. First, a layer in an RNN does not preserve the same feature latent space function for different inputs, making it impossible to map the inserted specific pattern with the neural activations. Second, the text data is inherently discrete, making it hard to optimize the text like image pixels. In this work, we propose a novel backdoor detection approach named InterRNN for RNN-based text classification systems from the interpretation perspective. Specifically, we first propose a novel RNN interpretation technique by constructing a nondeterministic finite automaton (NFA) based abstract model, which can effectively reduce the analysis complexity of an RNN while preserving its original logic rules. Then, based on the abstract model, we can obtain interpretation results that explain the fundamental reason behind the decision for each input. We then detect trigger words by leveraging the differences between the behaviors in the backdoor sentences and those in the normal sentences. The extensive experiment results on four benchmark datasets demonstrate that our approach can generate better interpretation results compared to state-of-the-art approaches and effectively detect backdoors in RNNs.
Keywords
Training, Recurrent neural networks, Task analysis, Motion pictures, Data models, Analytical models, Sentiment analysis, Text backdoor detection, RNN, model abstraction, interpretation
Discipline
OS and Networks | Software Engineering
Research Areas
Software and Cyber-Physical Systems
Publication
IEEE Transactions on Information Forensics and Security
Volume
16
First Page
4117
Last Page
4132
ISSN
1556-6013
Identifier
10.1109/TIFS.2021.3103064
Publisher
Institute of Electrical and Electronics Engineers
Citation
FAN, Ming; SI, Ziliang; XIE, Xiaofei; LIU, Yang; and LIU, Ting.
Text backdoor detection using an interpretable RNN abstract model. (2021). IEEE Transactions on Information Forensics and Security. 16, 4117-4132.
Available at: https://ink.library.smu.edu.sg/sis_research/7118