Abstract
We have designed a smart fridge that helps users to monitor and track the real-time status of the fridge and get recommendations on foods and recipes. The fridge recognizes every single food that users put in or retrieve out. On our App, users are able to see the real-time collections of current food inside the fridge with corresponding information including food image, weight, calories, expected expiration date as well as some physical stats inside the fridge like temperature and humidity. Also, there are some recommended recipes based on what food they currently have. We have overcome several challenges including sensor deployment, threads scheduling, image recognition, label parsing, database establishing for food nutrition information, and some difficulty in data visualization and synchronization. We have successfully created an intelligent and helpful smart fridge that performs well in the real-time environment and has fairly high food recognition accuracy.
Problem in a NutshellWith the significant development of IoT and deep learning, we are able to integrate these technologies more closely with our daily applications. For this project, our goal is building a smart refrigerator system which helps users to track the status of stored food and internal environment, and we can also provide some recommendations based on the current food.
To achieve this goal, we need to fulfill several system requirements: First, our system must be able to determine the category of the food that user puts into the refrigerator with image recognition. Then, we need to build an internal sensor node to collect and transmit the data which provide us the key information about the food and environment. Furthermore, we will process these data in the cloud, update them in real-time and get them ready for the queries from the users. Finally, it is necessary for us to create a mobile application for the user to conveniently monitor their fridge and get further information on their devices. |
Prior WorkThe pivotal part in our project is the food image recognition. The smart fridge must effectively and accurately recognize an arbitrary incoming food. This is essentially a deep learning problem. Over recent years, plenty of works have been done regarding image classification and recognition by deep neural network, including AlexNet, GoogleNet, VGGNet, ResNet, and etc[1]. Within our project, we apply AWS Rekognition[2], which is a powerful cloud service that performs great on visual analysis. The functionality of AWS Rekognition includes label detection of images, on which our project based.
|
Scenario
There are chicken, onions, carrots in the fridge. The app recommends a curry chicken for lunch to the user.
Bananas have been put inside for a week. User gets a reminding message about the coming expiration date.
|
Lucy keeps running and playing tennis on a regular basis but always forget to buy milk or eggs. The fridge sends her a notification to buy the food.
Thanksgiving Day’s coming, if there is no turkey in the fridge, the user is recommended to buy one.
|