Publication Type
Conference Proceeding Article
Version
publishedVersion
Publication Date
10-2019
Abstract
Fashion knowledge helps people to dress properly and addresses not only physiological needs of users, but also the demands of social activities and conventions. It usually involves three mutually related aspects of: occasion, person and clothing. However, there are few works focusing on extracting such knowledge, which will greatly benefit many downstream applications, such as fashion recommendation. In this paper, we propose a novel method to automatically harvest fashion knowledge from social media. We unify three tasks of occasion, person and clothing discovery from multiple modalities of images, texts and metadata. For person detection and analysis, we use the off-the-shelf tools due to their flexibility and satisfactory performance. For clothing recognition and occasion prediction, we unify the two tasks by using a contextualized fashion concept learning module, which captures the dependencies and correlations among different fashion concepts. To alleviate the heavy burden of human annotations, we introduce a weak label modeling module which can effectively exploit machine-labeled data, a complementary of clean data. In experiments, we contribute a benchmark dataset and conduct extensive experiments from both quantitative and qualitative perspectives. The results demonstrate the effectiveness of our model in fashion concept prediction, and the usefulness of extracted knowledge with comprehensive analysis.
Keywords
Fashion knowledge extraction, Fashion analysis
Discipline
Databases and Information Systems
Research Areas
Information Systems and Management
Publication
Proceedings of the 27th ACM International Conference on Multimedia, Nice, France, 2019 October 21 - 25
First Page
257
Last Page
265
ISBN
9781450368896
Identifier
10.1145/3343031.3350889
Publisher
Association for Computing Machinery
City or Country
Nice France
Citation
MA, Yunshan; YANG, Xun; LIAO, Lizi; CAO, Yixin; and CHUA, Tat-Seng.
Who, where, and what to wear?: extracting fashion knowledge from social media. (2019). Proceedings of the 27th ACM International Conference on Multimedia, Nice, France, 2019 October 21 - 25. 257-265.
Available at: https://ink.library.smu.edu.sg/sis_research/7571
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Additional URL
https://doi.org/10.1145/3343031.3350889