Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection
Publication Type
Conference Proceeding Article
Version
acceptedVersion
Publication Date
9-2022
Abstract
Current polyp detection methods from colonoscopy videos use exclusively normal (i.e., healthy) training images, which i) ignore the importance of temporal information in consecutive video frames, and ii) lack knowledge about the polyps. Consequently, they often have high detection errors, especially on challenging polyp cases (e.g., small, flat, or partially visible polyps). In this work, we formulate polyp detection as a weakly-supervised anomaly detection task that uses video-level labelled training data to detect frame-level polyps. In particular, we propose a novel convolutional transformer-based multiple instance learning method designed to identify abnormal frames (i.e., frames with polyps) from anomalous videos (i.e., videos containing at least one frame with polyp). In our method, local and global temporal dependencies are seamlessly captured while we simultaneously optimise video and snippet-level anomaly scores. A contrastive snippet mining method is also proposed to enable an effective modelling of the challenging polyp cases. The resulting method achieves a detection accuracy that is substantially better than current state-of-the-art approaches on a new large-scale colonoscopy video dataset introduced in this work. Our code and dataset are available at https://github.com/tianyu0207/weakly-polyp.
Keywords
Polyp detection, Colonoscopy, Weakly-supervised learning, Video anomaly detection, Vision transformer
Discipline
Artificial Intelligence and Robotics | Medical Sciences
Research Areas
Data Science and Engineering
Publication
Proceedings of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, Singapore, 2022 September 18 - 22
Volume
13433
ISBN
9783031164361
Identifier
10.1007/978-3-031-16437-8_9
Publisher
Springer Nature Switzerland
City or Country
Switzerland
Citation
YU, Tian; PANG, Guansong; LIU, Fengbei; LIU, Yuyuan; WANG, Chong; CHEN, Yuanhong; VERJANS, Johan; and CARNEIRO, Gustavo.
Contrastive transformer-based multiple instance learning for weakly supervised polyp frame detection. (2022). Proceedings of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, Singapore, 2022 September 18 - 22. 13433,.
Available at: https://ink.library.smu.edu.sg/sis_research/7549
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1007/978-3-031-16437-8_9