Publication Type
Journal Article
Version
acceptedVersion
Publication Date
5-2024
Abstract
Nighttime semantic segmentation is an important but challenging research problem for autonomous driving. The major challenges lie in the small objects or regions from the under-/over-exposed areas or suffer from motion blur caused by the camera deployed on moving vehicles. To resolve this, we propose a novel hard- class-aware module that bridges the main network for full-class segmentation and the hard-class network for segmenting aforementioned hard-class objects. In specific, it exploits the shared focus of hard-class objects from the dual-stream network, enabling the contextual information flow to guide the model to concentrate on the pixels that are hard to classify. In the end, the estimated hard-class segmentation results will be utilized to infer the final results via an adaptive probabilistic fusion refinement scheme. Moreover, to overcome over- smoothing and noise caused by extreme exposures, our model is modulated by a carefully crafted pretext task of constructing an exposure-aware semantic gradient map, which guides the model to faithfully perceive the structural and semantic information of hard-class objects while mitigating the negative impact of noises and uneven exposures. In experiments, we demonstrate that our unique network design leads to superior segmentation performance over existing methods, featuring the strong ability of perceiving hard-class objects under adverse conditions.
Keywords
Nighttime semantic segmentation, dual-branch, hard class, pretext task, fusion refinement scheme
Discipline
Artificial Intelligence and Robotics | Graphics and Human Computer Interfaces | Transportation
Research Areas
Intelligent Systems and Optimization
Publication
ACM Transactions on Multimedia Computing, Communications and Applications
Volume
20
Issue
7
First Page
1
Last Page
23
ISSN
1551-6857
Identifier
10.1145/3650032
Publisher
Association for Computing Machinery (ACM)
Citation
LIU, Wenxi; CAI, Jiaxin; LI, Qi; LIAO, Chenyang; CAO, Jingjing; HE, Shengfeng; and YU, Yuanlong.
Learning nighttime semantic segmentation the hard way. (2024). ACM Transactions on Multimedia Computing, Communications and Applications. 20, (7), 1-23.
Available at: https://ink.library.smu.edu.sg/sis_research/9801
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3650032
Included in
Artificial Intelligence and Robotics Commons, Graphics and Human Computer Interfaces Commons, Transportation Commons
