Exploring structural knowledge for automated visual inspection of moving trains
Publication Type
Journal Article
Publication Date
2-2022
Abstract
Deep learning methods are becoming the de-facto standard for generic visual recognition in the literature. However, their adaptations to industrial scenarios, such as visual recognition for machines, product streamlines, etc., which consist of countless components, have not been investigated well yet. Compared with the generic object detection, there is some strong structural knowledge in these scenarios (e.g., fixed relative positions of components, component relationships, etc.). A case worth exploring could be automated visual inspection for trains, where there are various correlated components. However, the dominant object detection paradigm is limited by treating the visual features of each object region separately without considering common sense knowledge among objects. In this article, we propose a novel automated visual inspection framework for trains exploring structural knowledge for train component detection, which is called SKTCD. SKTCD is an end-to-end trainable framework, in which the visual features of train components and structural knowledge (including hierarchical scene contexts and spatial-aware component relationships) are jointly exploited for train component detection. We propose novel residual multiple gated recurrent units (Res-MGRUs) that can optimally fuse the visual features of train components and messages from the structural knowledge in a weighted-recurrent way. In order to verify the feasibility of SKTCD, a dataset that contains high-resolution images captured from moving trains has been collected, in which 18 590 critical train components are manually annotated. Extensive experiments on this dataset and on the PASCAL VOC dataset have demonstrated that SKTCD outperforms the existing challenging baselines significantly. The dataset as well as the source code can be downloaded online (https://github.com/smartprobe/SKCD).
Keywords
Object detection, Visualization, Fasteners, Feature extraction, Inspection, Proposals, Wheels, Automated visual inspection, deep convolutional neural networks (DCNNs), object detection, train component detection
Discipline
Artificial Intelligence and Robotics | Databases and Information Systems
Research Areas
Data Science and Engineering; Intelligent Systems and Optimization
Publication
IEEE Transactions on Cybernetics
Volume
52
Issue
2
First Page
1233
Last Page
1246
ISSN
2168-2267
Identifier
10.1109/TCYB.2020.2998126
Publisher
Institute of Electrical and Electronics Engineers
Citation
CHEN, Cen; ZOU, Xiaofeng; ZENG, Zeng; CHENG, Zhongyao; ZHANG, Le; and HOI, Steven C. H..
Exploring structural knowledge for automated visual inspection of moving trains. (2022). IEEE Transactions on Cybernetics. 52, (2), 1233-1246.
Available at: https://ink.library.smu.edu.sg/sis_research/7242
Additional URL
https://doi.org/10.1109/TCYB.2020.2998126