Publication Type
Journal Article
Version
acceptedVersion
Publication Date
9-2024
Abstract
This paper addresses the significant challenges in 3D Semantic Scene Graph (3DSSG) prediction, essential for understanding complex 3D environments. Traditional approaches, primarily using PointNet and Graph Convolutional Networks, struggle with effectively extracting multi-grained features from intricate 3D scenes, largely due to a focus on global scene processing and single-scale feature extraction. To overcome these limitations, we introduce Granular3D, a novel approach that shifts the focus towards multi-granularity analysis by predicting relation triplets from specific sub-scenes. One key is the Adaptive Instance Enveloping Method (AIEM), which establishes an approximate envelope structure around irregular instances, providing shape-adaptive local point cloud sampling, thereby comprehensively covering the contextual environments of instances. Moreover, Granular3D incorporates a Hierarchical Dual-Stage Network (HDSN), which differentiates and processes features of instances and their pairs at varying scales, leading to a targeted prediction of instance categories and their relationships. To advance the perception of sub-scene in HDSN, we design a Gather Point Transformer structure (GaPT) that enables the combinatorial interaction of local information from multiple point cloud sets, achieving a more comprehensive local contextual feature extraction. Extensive evaluations on the challenging 3DSSG benchmark demonstrate that our methods provide substantial improvements, establishing a new state-of-the-art in 3DSSG prediction, boosting the top-50 triplet accuracy by ＋2.8%.
Keywords
3D point cloud, 3D semantic scene graph prediction, Gather point transformer, Multi-granularity
Discipline
Graphics and Human Computer Interfaces | Software Engineering
Research Areas
Software and Cyber-Physical Systems
Publication
Pattern Recognition
Volume
153
First Page
1
Last Page
12
ISSN
0031-3203
Identifier
10.1016/j.patcog.2024.110562
Publisher
Elsevier
Citation
HUANG, Kaixiang; YANG, Jingru; WANG, Jin; HE, Shengfeng; WANG, Zhan; HE, Haiyan; ZHANG, Qifeng; and LU, Guodong.
Granular3D: Delving into multi-granularity 3D scene graph prediction. (2024). Pattern Recognition. 153, 1-12.
Available at: https://ink.library.smu.edu.sg/sis_research/8811
Copyright Owner and License
Authors
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1016/j.patcog.2024.110562