Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
8-2023
Abstract
Recent semi-supervised anomaly detection methods that are trained using small labeled anomaly examples and large unlabeled data (mostly normal data) have shown largely improved performance over unsupervised methods. However, these methods often focus on fitting abnormalities illustrated by the given anomaly examples only (i.e., seen anomalies), and consequently they fail to generalize to those that are not, i.e., new types/classes of anomaly unseen during training. To detect both seen and unseen anomalies, we introduce a novel deep weakly-supervised approach, namely Pairwise Relation prediction Network (PReNet), that learns pairwise relation features and anomaly scores by predicting the relation of any two randomly sampled training instances, in which the pairwise relation can be anomaly-anomaly, anomaly-unlabeled, or unlabeled-unlabeled. Since unlabeled instances are mostly normal, the relation prediction enforces a joint learning of anomaly-anomaly, anomaly-normal, and normal-normal pairwise discriminative patterns, respectively. PReNet can then detect any seen/unseen abnormalities that fit the learned pairwise abnormal patterns, or deviate from the normal patterns. Further, this pairwise approach also seamlessly and significantly augments the training anomaly data. Empirical results on 12 real-world datasets show that PReNet significantly outperforms nine competing methods in detecting seen and unseen anomalies. We also theoretically and empirically justify the robustness of our model w.r.t. anomaly contamination in the unlabeled data. The code is available at https://github.com/mala-lab/PReNet.
Keywords
Anomaly detection, Anomaly detection methods, Deep learning; Intrusion-Detection, Learn+, Performance, Semi-supervised, Type class, Unlabeled data, Unsupervised method
Discipline
Databases and Information Systems
Research Areas
Data Science and Engineering; Information Systems and Management
Publication
Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach, United States, 2023 August 6-10
First Page
1795
Last Page
1807
ISBN
9798400701030
Identifier
10.1145/3580305.3599302
Publisher
Association for Computing Machinery
City or Country
New York
Citation
PANG, Guansong; SHEN, Chunhua; JIN, Huidong; and VAN DEN HENGEL, Anton.
Deep weakly-supervised anomaly detection. (2023). Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach, United States, 2023 August 6-10. 1795-1807.
Available at: https://ink.library.smu.edu.sg/sis_research/8411
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3580305.3599302