How to build hierarchical classification for image recognition

Author(s): Michal Lukac

Originally published on Towards AI.

The game-changing feature that makes deep learning more accessible. Solving computer vision problem of Screw & Nut recognition with hierarchical classification.

Nowadays, there are a lot of companies that are doing AI/Machine Learning. For example, many blue-chip companies are integrating their own machine learning solutions to their cloud. Using their cloud is becoming more and more complex and people started to become XYZ cloud service CERTIFIED specialists in order to use it.

“Everything should be made as simple as possible, but no simpler.”

Albert Einstein

Sometimes you or your team need to build accurate and more complex machine learning systems. But this is very hard when using AWS, Azure or AutoML consoles. The cloud solutions from big players have so many buttons, pages, and settings that it is very hard to use them. It is even harder if you are building a model from scratch by yourself coding in frameworks like PyTorch and TensorFlow as you need to have some coding and machine learning skills …

People love to use Apple products because Apple puts complex things into easy to use devices.

So what should I do if I’m working on something complex?

My simple advice is that every complex problem can be divided into more simple subproblems. However, the step of connecting simple problems back to the working solution can be tricky, expensive and take a lot of time.

Today I would like to present a game-changing feature on building hierarchical machine learning systems with Ximilar Flows. We build many successful products with it for fashion and healthcare.

And now you can too in another or the same domains!

Today we will go for a use case more focused on image recognition. If you never trained an AI vision model through the Ximilar App then I recommend reading my previous post:

How to detect defects on images

Building a visual quality control system to check your products on a few clicks with Convolutional Neural Networks and…

medium.com

What is a hierarchical classification?

Imagine the problem of categorization image to thousand of classes….

The idea of hierarchical classification is pretty old and simple. Instead of using one model which is predicting everything, you could utilize more models and connect them in such a way that you will get the desired results.

So the first model in the hierarchy can recognize Dog, Bird, … If the model predicts that on the picture is Bird then the separate model for Bird subtype is called to distinguish Parrot, Pidgeon or Eagle. Of course, this is just a simple hierarchy but you can imagine that it can be wider and deeper.

More advanced use case …

There are thousands of screws & nuts which differ in size, head type, material. Building the system which is correctly assigning an image to thousands of categories can be unbelievable hard. It could take you a lot of time and development. We would need also a lot of images/samples and high-quality labels to train such a model.

Instead of one model, we could create the first model to recognize Nut vs Screw. If we recognize Nut we can then call the model to get the type of the Nut => Hex, Wing, Square. If Screw is detected then identify Material, Head Type and Size. The model for the material will tell you if the screw is made of steel or zinc. The overview of the system can be visualized in this way:

We are going to create our 5 image recognition tasks/models through the Ximilar App platform. Just upload images to the labels, create tasks and connect labels to the tasks. Finally, you need to train them!

And now the mind-blowing part!

We will connect all the tasks through the flow. As the first step, we create a branch selector action and set the branching task to “Screw vs Nut”. The output on the query image from this task is stored in the “Category” field of JSON when querying the API endpoint. Based on the result from the branch the system will choose one of the branches and call other tasks.

Now you can test it through the Test Flows page or call the API endpoint. Your flow can be much more complex. Some of our internal flows contain nested branches with hundreds of tasks and labels.

More examples…

Here are just some of the examples of how you can use the flow:

• Real Estate — recognize room type and then get the home features and overall condition quality of the house
• Industries & Manufacturing— recognize an object and tell if it is the defect or not and what part of the object has defected if so
• Microscopy imaging — recognize a complex hierarchy of bacterias, …
• X-Ray dental images — detect teeth and identify problems, restoration, …

Summarize

Ximilar Flows system is very flexible and scalable to all other domains (real estate, manufacturing, health care…). We think this will help your business to create machine learning products faster. It does not matter if you are a freelancer, startup or big corporation.

We have more plans on flows with the integration of object detection and segmentation soon! So you will be able to detect an object (exact location) for example of Bird and then calling your categorization task on this object to tell which species of bird it is.

Without a single line of code, we are making machine learning more accessible to everyone!

For more news and tips -> follow us on Twitter or LinkedIn!

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI