Learning long-term structural dependencies for video salient object detection

Publication Type

Journal Article

Publication Date

1-2020

Abstract

Existing video salient object detection (VSOD) methods focus on exploring either short-term or long-term temporal information. However, temporal information is exploited in a global frame-level or regular grid structure, neglecting inter-frame structural dependencies. In this article, we propose to learn long-term structural dependencies with a structure-evolving graph convolutional network (GCN). Particularly, we construct a graph for the entire video using a fast supervoxel segmentation method, in which each node is connected according to spatio-temporal structural similarity. We infer the inter-frame structural dependencies of salient object using convolutional operations on the graph. To prune redundant connections in the graph and better adapt to the moving salient object, we present an adaptive graph pooling to evolve the structure of the graph by dynamically merging similar nodes, learning better hierarchical representations of the graph. Experiments on six public datasets show that our method outperforms all other state-of-the-art methods. Furthermore, We also demonstrate that our proposed adaptive graph pooling can effectively improve the supervoxel algorithm in the term of segmentation accuracy.

Keywords

Predictive models, Object detection, Feature extraction, Saliency detection, Convolution, Merging, Object recognition, Video salient object detection, graph convolutional network, supervoxel

Discipline

Information Security

Research Areas

Information Systems and Management

Publication

IEEE Transactions on Image Processing

Volume

29

First Page

9017

Last Page

9031

ISSN

1057-7149

Identifier

10.1109/TIP.2020.3023591

Publisher

Institute of Electrical and Electronics Engineers

Additional URL

https://doi.org/10.1109/TIP.2020.3023591

This document is currently not available here.

Share

COinS