Boundary-aware RGBD salient object detection with cross-modal feature sampling
Publication Type
Journal Article
Publication Date
1-2020
Abstract
Mobile devices usually mount a depth sensor to resolve ill-posed problems, like salient object detection on cluttered background. The main barrier of exploring RGBD data is to handle the information from two different modalities. To cope with this problem, in this paper, we propose a boundary-aware cross-modal fusion network for RGBD salient object detection. In particular, to enhance the fusion of color and depth features, we present a cross-modal feature sampling module to balance the contribution of the RGB and depth features based on the statistics of their channel values. In addition, in our multi-scale dense fusion network architecture, we not only incorporate edge-sensitive losses to preserve the boundary of the detected salient region, but also refine its structure by merging the estimated saliency maps of different scales. We accomplish the multi-scale saliency map merging using two alternative methods which produce refined saliency maps via per-pixel weighted combination and an encoder-decoder network. Extensive experimental evaluations demonstrate that our proposed framework can achieve the state-of-the-art performance on several public RGBD-based datasets.
Keywords
Salient object detection, cross-modal, boundary-aware estimation
Discipline
Information Security
Research Areas
Information Systems and Management
Publication
IEEE Transactions on Image Processing
Volume
29
First Page
9496
Last Page
9507
ISSN
1057-7149
Identifier
10.1109/TIP.2020.3028170
Publisher
Institute of Electrical and Electronics Engineers
Citation
NIU, Yuzhen; LONG, Guanchao; LIU, Wenxi; GUO, Wenzhong; and HE, Shengfeng.
Boundary-aware RGBD salient object detection with cross-modal feature sampling. (2020). IEEE Transactions on Image Processing. 29, 9496-9507.
Available at: https://ink.library.smu.edu.sg/sis_research/7847
Additional URL
https://doi.org/10.1109/TIP.2020.3028170