Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

11-2019

Abstract

We present SmrtFridge, a consumer-grade smart fridge prototype that demonstrates two key capabilities: (a) identify the individual food items that users place in or remove from a fridge, and (b) estimate the residual quantity of food items inside a refrigerated container (opaque or transparent). Notably, both of these inferences are performed unobtrusively, without requiring any explicit user action or tagging of food objects. To achieve these capabilities, SmrtFridge uses a novel interaction-driven, multi-modal sensing pipeline, where Infrared (IR) and RGB video sensing, triggered whenever a user interacts naturally with the fridge, is used to extract a foreground visual image of the food item, which is then processed by a state-of-the-art DNN classifier. Concurrently, the residual food quantity is estimated by exploiting slight thermal differences, between the empty and filled portions of the container. Experimental studies, involving 12 users interacting naturally with 19 common food items and a commodity fridge, show that SmrtFridge is able to (a) extract at least 75% of a food item's image in over 97% of interaction episodes, and consequently identify the individual food items with precision/recall values of ~ 85%, and (b) perform robust coarse-grained (3 level) classification of the residual food quantity with an accuracy of ~ 75%.

Keywords

Object Segmentation, IR sensing, Internet of Things (IoT)

Discipline

Health Information Technology | Software Engineering | Technology and Innovation

Research Areas

Software and Cyber-Physical Systems

Publication

SenSys '19: Proceedings of the 17th Conference on Embedded Networked Sensor Systems, New York, November 10-13

First Page

245

Last Page

257

ISBN

9781450369503

Identifier

10.1145/3356250.3360028

Publisher

ACM

City or Country

New York

Copyright Owner and License

Publisher

Additional URL

https://doi.org/10.1145/3356250.3360028

Download

Share

COinS