Publication Type
Journal Article
Version
acceptedVersion
Publication Date
12-2023
Abstract
Isolation forest (iForest) has been emerging as arguably the most popular anomaly detector in recent years due to its general effectiveness across different benchmarks and strong scalability. Nevertheless, its linear axis-parallel isolation method often leads to (i) failure in detecting hard anomalies that are difficult to isolate in high-dimensional/non-linear-separable data space, and (ii) notorious algorithmic bias that assigns unexpectedly lower anomaly scores to artefact regions. These issues contribute to high false negative errors. Several iForest extensions are introduced, but they essentially still employ shallow, linear data partition, restricting their power in isolating true anomalies. Therefore, this paper proposes deep isolation forest. We introduce a new representation scheme that utilises casually initialised neural networks to map original data into random representation ensembles, where random axis-parallel cuts are subsequently applied to perform the data partition. This representation scheme facilitates high freedom of the partition in the original data space (equivalent to non-linear partition on subspaces of varying sizes), encouraging a unique synergy between random representations and random partition-based isolation. Extensive experiments show that our model achieves significant improvement over state-of-the-art isolation-based methods and deep detectors on tabular, graph and time series datasets; our model also inherits desired scalability from iForest.
Keywords
Anomaly Detection, Isolation Forest, Deep Representation, Ensemble Learning
Discipline
Databases and Information Systems | Theory and Algorithms
Research Areas
Data Science and Engineering
Publication
IEEE Transactions on Knowledge and Data Engineering
Volume
35
Issue
12
First Page
12591
Last Page
12604
ISSN
1041-4347
Identifier
10.1109/TKDE.2023.3270293
Publisher
Institute of Electrical and Electronics Engineers
Citation
XU, Hongzuo; PANG, Guansong; WANG, Yijie; and WANG, Yongjun.
Deep isolation forest for anomaly detection. (2023). IEEE Transactions on Knowledge and Data Engineering. 35, (12), 12591-12604.
Available at: https://ink.library.smu.edu.sg/sis_research/8003
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1109/TKDE.2023.3270293