Publication Type

Journal Article

Version

acceptedVersion

Publication Date

2-2024

Abstract

Food computing has long been studied and deployed to several applications. Understanding a food image at the instance level, including recognition, counting and segmentation, is essential to quantifying nutrition and calorie consumption. Nevertheless, existing techniques are limited to either category-specific instance detection, which does not reflect precisely the instance size at the pixel level, or category-agnostic instance segmentation, which is insufficient for dish recognition. This paper presents a compact and fast multi-task network, namely FoodMask, for clustering-based food instance counting, segmentation and recognition. The network learns a semantic space simultaneously encoding food category distribution and instance height at pixel basis. While the former value addresses instance recognition, the latter value provides prior knowledge for instance extraction. Besides, we integrate into the semantic space a pathway for class-specific counting. With these three outputs, we propose a clustering algorithm to segment and recognize food instances at a real-time speed. Empirical studies are made on three large-scale food datasets, including Mixed Dishes, UECFoodPixComp and FoodSeg103, which cover Western, Chinese, Japanese and Indian cuisines. The proposed networks outperform benchmarks in both terms of instance map quality and speed efficiency.

Keywords

Food counting, Food instance segmentation, Food recognition

Discipline

Databases and Information Systems | Food Science | Graphics and Human Computer Interfaces

Research Areas

Software and Cyber-Physical Systems

Publication

Pattern Recognition

Volume

146

First Page

1

Last Page

11

ISSN

0031-3203

Identifier

10.1016/j.patcog.2023.110017

Publisher

Elsevier

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1016/j.patcog.2023.110017

Download

Share

COinS