Calculating Nutrition Facts with Computer Vision — Foodify.ai

Author(s): Durgesh Samariya

Originally published on Towards AI the World’s Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses.

Computer Vision

Calculating Nutrition Facts with Computer Vision — Foodify.ai

A Computer Vision-based Indian Food Detection and Nutrition Calculation App

The second most populous country in the world, India has enjoyed steady economic growth and has achieved self-sufficiency in grain production in recent years. — https://www.wfp.org/countries/india

Indian cuisine consists of a variety of traditional cuisines native to the region. Given the diversity in soil, climate, culture, ethnic groups, and occupations, these cuisines vary substantially and use locally available spices, herbs, vegetables, and fruits. Around 1.4 Billion people leaving in 36 states and union territories, with unique dishes and heritage.

In this era, people are becoming more health-conscious than before. However, there is a lack of knowledge about different fitness and wellness aspects of food. Thus, I come up with Foodify.ai — a deep learning-based application that detects food from the image and provides information of food such as protein, vitamins, calories, minerals, carbs, etc.

As far as I know, there is no such application or public dataset available in the context of Indian food for the deep learning community. Thus, to address this research gap, I developed Foodify.ai. The goal of this app is:

1. a world’s largest Indian food image dataset (I will make it publicly available)
2. an app that detects Indian food and provides nutrition information.

Stages of application

Stage 1: Image Collection Application (✅)

To develop any machine learning or deep learning model, the most important thing is data. So to start with, I downloaded images from Google and Microsoft bing. However, to create the world’s largest dataset these images are not enough. So, I created a web app that collects images from users. If you want to contribute, you can upload images here.

Stage 2: Train Deep Learning Model and Develop Prototype Application

Currently, I have trained a model using 30 different food items. You can check the working application here.

Stage 3: Create a nutrition Database

In this, I will create a nutrition database by searching different nutrition databases. So far I am aware of only USDA Nutritional Database. However, I will search more when I will start this step.

Stage 4: Develop Mobile App

To develop a mobile app, I have to integrate whatever I have done in stages 1, 2, and 3. The final mobile application will allow a user to click/upload an image of any Indian food, which is passed as input to our model and return food type. From the detected food type, our app will look in the nutrition database and finally return information related to that food from our nutrition data.

Challenges

The task of food detection/classification is not easy as it seems. During my early days, my primary goal was to create the world’s largest Indian food dataset for my research. Initially, I defined 151 Food categories to create my model. However, it is difficult to distinguish some items such as Naan, Butter Naan, Cheese Naan, and Garlic Naan. Similarly, it is difficult to identify Dal, Dal Fry, Dal Tadka, or let’s consider Paratha, Aloo Paratha, Gobi Paratha, and Paneer Paratha. Thus, I created a single category for Naan, Dal, Parotha, and many more. In the final version, I am planning to return all possible options related to the given Image. For example, if a user uploads a dal image then the Foodify.ai app return all dal’s from our nutrition database such as Dal Tadka, Dal Fry, Dal Makhni, etc.

Another challenge is to collect images to create a huge dataset. To solve this issue I have created a web app for image collection which you can find here. I am also planning to crawl images from Instagram, Flicker, Pinterest, and other social media platforms for images for each category.

There is another challenge, which is related to training the deep learning model. It is an extremely computationally expensive and time-consuming task to train the model again and again. This can be solved by using cloud-based services. (Already using AWS).

You can find code at my GitHub.

I highly appreciate your help in making this dataset.

About Me

I am Durgesh Samariya, a 3rd-year Machine Learning Ph.D. student at FedUni, Australia. On the internet, I am known as TheMLPhDStudent.

On the Internet:

Follow me on Instagram, Kaggle, GitHub, and Medium.

Thanks for reading.

Calculating Nutrition Facts with Computer Vision — Foodify.ai was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Join thousands of data leaders on the AI newsletter. It’s free, we don’t spam, and we never share your email address. Keep up to date with the latest work in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI