DeepFacade: A deep learning approach to facade parsing with symmetric loss

Publication Type

Journal Article

Publication Date

12-2020

Abstract

Parsing building facades into procedural grammars plays an important role for 3D building model generation tasks, which have been long desired in computer vision. Deep learning is a promising approach to facade parsing, however, a straightforward solution by directly applying standard deep learning approaches cannot always yield the optimal results. This is primarily due to two reasons: 1) it is nontrivial to train existing semantic segmentation networks for facade parsing, e.g., Fully-Convolutional Neural Networks (FCN) which are usually weak at predicting fine-grained shapes (J. Long et al., 2015); and 2) building facades are man-made architectures with highly regularized shape priors, and the prior knowledge plays an important role in facade parsing, for which how to integrate the prior knowledge into deep neural networks remains an open problem. In this paper, we present a novel symmetric loss function that can be used in deep neural networks for end-to-end training. This novel loss is based on the assumption that most of windows and doors have a highly symmetric rectangle shape, and it penalizes all window predictions that are non-rectangles. This prior knowledge is smoothly integrated into the end-to-end training process. Quantitative evaluation demonstrates that our method has outperformed previous state-of-art methods significantly on five popular facade parsing datasets. Qualitative results have shown that our method effectively aids deep convolutional neural networks to predict more accurate, visually pleasing, and symmetric shapes. To the best of our knowledge, we are the first to incorporate symmetry constraint into end-to-end training in deep neural networks for facade parsing.

Keywords

Shape;Windows;Microsoft Windows;Grammar;Deep learning;Semantics;Facade parsing;deep learning;semantic segmentation

Discipline

Databases and Information Systems

Research Areas

Data Science and Engineering

Publication

IEEE Transactions on Multimedia

Volume

22

Issue

12

First Page

3153

Last Page

3165

ISSN

1520-9210

Identifier

10.1109/TMM.2020.2971431

Publisher

Institute of Electrical and Electronics Engineers

Additional URL

https://doi.org/10.1109/TMM.2020.2971431

This document is currently not available here.

Share

COinS