Publication Type

Journal Article

Version

acceptedVersion

Publication Date

1-2022

Abstract

Cross-modal recipe retrieval has recently been explored for food recognition and understanding. Text-rich recipe provides not only visual content information (e.g., ingredients, dish presentation) but also procedure of food preparation (cutting and cooking styles). The paired data is leveraged to train deep models to retrieve recipes for food images. Most recipes on the Web include sample pictures as the references. The paired multimedia data is not noise-free, due to errors such as pairing of images containing partially prepared dishes with recipes. The content of recipes and food images are not always consistent due to free-style writing and preparation of food in different environments. As a consequence, the effectiveness of learning cross-modal deep models from such noisy web data is questionable. This paper conducts an empirical study to provide insights whether the features learnt with noisy pair data are resilient and could capture the modality correspondence between visual and text.

Keywords

Image recognition;Training;Generative adversarial networks;Feature extraction;Visualization;Data models;Context modeling;Food recognition;image-to-recipe retrieval;image-to-image retrieval

Discipline

Databases and Information Systems

Research Areas

Data Science and Engineering

Publication

IEEE Transactions on Multimedia

Volume

24

First Page

1175

Last Page

1185

ISSN

1520-9210

Identifier

10.1109/TMM.2021.3123474

Publisher

Institute of Electrical and Electronics Engineers

Additional URL

https://doi.org/10.1109/TMM.2021.3123474

Download

Share

COinS