Teach you how to build a food recognition AI step by step: novices can easily get started, popular and highly praised with demo | Resources

Compiled and edited by Tong Ling
Produced by Quantum Bit | Public Account QbitAI

Who says deep learning tasks are complicated and difficult to understand?

In fact, even a novice can easily get started and create interesting and useful works.

Recently, engineers Nidhin Pattaniyil and Reshama Shaikh wrote a tutorial on building a web & app version of food recognition AI from scratch. The tutorial includes data, tutorials, and demos. It is concise and easy to understand, and has won a large number of likes on Twitter.

Even Jeremy Howard, founder of fast.ai and former chairman and chief scientist of Kaggle, gave a thumbs up and recommended it:

Let’s take a look at the detailed steps below:

warm up

First, let’s take a look at the task, and then use deep learning methods to build food image classifiers on the web and mobile terminals.

After completing this project, you will gain the following skills:

• Use the fastai library to train a deep learning model for food photo classification

• Deploy this model to the web and mobile using Heroku and Flask

This tutorial is divided into three parts, the directory is as follows:

Part 1 : Training the Classifier

Part 2 : Deploy the model to the web page

Part 3 : Deploy the model to the mobile terminal

Without further ado, let’s start by explaining the specific steps.

Training the classifier

To build such a fun application, you need to first obtain the model weight file , which you can obtain from any deep learning library. The two authors used the fastai library.

Open the following GitHub repo, where the Jupyter Notebook code can help you train the model.

Address:
https://github.com/npatta01/web-deep-learning-classifier

In addition, you need to prepare relevant software for model training. The two guys used:

• fasti: version 1.0.42

• PyTorch: version 1.0.0

• Python: version 3.6

Version 1.0 of the deep learning fastai library is available and needs to be run on PyTorch. If you are not familiar with the operation of fastai, there is a set of fastai courses to learn:

https://docs.fast.ai/

The data we use is retrieved and analyzed on the Google Cloud Platform, but you can choose any cloud platform (Amazon, Paperspce, etc.).

You can find instructions for installing a GPU working environment on fastai. The equipment used by the two guys in this project is an NVIDIA Tesla P100 and a 100GB hard drive.

The guy also recommended an interesting food classification dataset: Food-101.

The Food-101 dataset contains 101 categories of food, with a total of 101k (that is, 101,000) images. There are 1,000 images in each category, of which 250 are test images and 750 are training images.

However, it should be noted that the training images in this dataset are not clean and contain some noisy data, and some labels are wrong.

In this dataset, all images have been resized to a maximum side length of 512 pixels. The dataset is 5GB in size and can be retrieved using the following code:

Open the tar.gz file with the following code block:

With the hardware and software ready and the data set ready, we can start training. We used the Resnet-34 CNN architecture to train the model, which took about an hour to run on the Google Cloud Platform.

The code for training the model can be found at the following address:

https://github.com/npatta01/web-deep-learning-classifier

Read these images using ImageDataBunch:

This data has been pre-trained on resnet34, and now we just need to run it directly to determine the best learning rate.

In this case, learning rate = 0.01 works best:

The researchers also ran more complex architectures, but ultimately settled on a simple Resnet34 architecture, given the 101 categories in the data and the processing time and storage constraints of the GPU.

Then, after 8 iterations with a learning rate of 0.01, the error rate was reduced to 28.9%.

Then, the guy unfroze the last few layers of the network and looked for a new learning rate.

After 5 iterations, the error rate decreased from 28.9% to 28.1%.

The confusion matrix shows that the following categories of food are frequently misclassified:

Output deep learning model

The output of this deep learning model is accompanied by a weight file, model.pth (or final.pth). If you follow this tutorial to train, the model is stored in the models code block.

The final.pth file can be downloaded from Jupyter to your computer.

The model.pth file may be too large to be included in the git commit. Here are some solutions:

1. Store in Google Drive

2. Store in GitHub releases

3. Store in a bucket on the cloud

The two foreign guys stored their models in GitHub releases, address:

https://github.com/npatta01/web-deep-learning-classifier/releases

Deploy on the web

Now that the model is trained, we can finally start the deployment phase. I use this repo as a template:

https://github.com/npatta01/web-deep-learning-classifier

And use the output of the classifier algorithm model.pth as the input of the Heroku app.

Required file updates

If you want to run this web application, you must first do the following:

1. Update the following content in the src/config.yaml file:

• title

• description

• about

• code

• sampleImages

2. Change the application name in the docs/2_heroku_app.md file to APP_NAME="food-img-classifier.

Install Heroku

If you don't have a Heroku account, sign up first (www.heroku.com).

Remember to update APP_NAME in your code:

Note: If there is no operation within 15 minutes, Heroku will be suspended. When the web application is woken up again, Heroku will be woken up again.

Deploy the model on mobile devices

First send the repo address:

https://github.com/npatta01/mobile-deep-learning-classifier

Required file updates

If you want to run this application on your phone, you need to do the following:

In the config.js file, update the following code block:

• AppConfig

• update the title

• point host to the heroku app url

• description

The guy also posted a demo of the food classification application on the mobile phone, the address is:

https://www.youtube.com/watch?reload=9&v=7d2qFLeYvRc

Cost Estimation

Finally, the guy estimated the economic costs required to complete these operations, including:

1. Apple store developer account registration fee: $100

2. Android Google Play registration fee: $30

Okay~ that’s all for the detailed tutorial, you can start now.

Portal

Tutorial:
https://reshamas.github.io/deploying-deep-learning-models-on-web-and-mobile/

Web version Demo address:
https://food-img-classifier.herokuapp.com/

iOS version Demo address:
https://itunes.apple.com/us/app/food-classifier/id1445356461?mt=8

Android version Demo address:
https://play.google.com/store/apps/details?id=com.rsnp.foodclassifier