Understanding the IoU Metric in Object Detection

Author(s): Pushkar Pushp

Originally published on Towards AI.

Computer Vision

Introduction

In this post, we will cover the metric used for the evaluation of the object detection model. The metric is invariant of algorithms whether one uses RCNN, Fast-RCNN, Faster- RCNN, YOLO, etc.

The blog will be primarily divided into three sections, the first one covering 'what', ‘why' is IoU needed. The second one will be python implementation and finally wrapping up with its application in the context of bounding box selection via non-max suppression.

Once you build an object detector next thing you want to know is how accurate is your model. This is common across all machine learning projects that before implementation, one should evaluate the model. In the case of regression problems we have a metric like Mean Square Error (MSE), similarly, for classification one can use confusion matrix,f1 score, etc; refer to this blog for detail.

Now coming back to our problem, the task of object detection or output of object detection is nothing but draw a bounding box with a certain probability.

The bounding box is a set of coordinates to use to localize image, it varies for COCO and Pascal-VOC datasets.

• COCO: bbox is (top left x position, top left y position, width, height)
• Pascal-VOC:bbox is (xmin-top left, ymin-top left,xmax-bottom right, ymax-bottom right)

Further details about these datasets can be read here.

Red Box: Bounding box from prediction.

Black Box: Actual bounding box or ground truth bounding box.

Unlike other metrics such as accuracy, f1 score, here we use the concept of similarity instead of point match of bbox. One such measure of similarity is the Jaccard index or in the colloquial language of computer vision, we refer it as intersection over the union.

Suppose we have two sets S1 and S2, Jaccard index J is defined as

J(S1, S2) = U+007CS1 ∩ S2U+007C/ U+007CS1 U S2U+007C

IoU = Area of INTERSECTION /Area of UNION.

• IoU score ≥0.5 is considered as good.

Python Implementation

box1 = (21, 11, 14, 13)
box2 = (23, 12, 13, 14) 
print("iou = " + str(iou(box1, box2)))

Output : iou = 0.417

Some other applications of the IoU metric.

Non-max suppression

Once you have fixed a threshold all bounding box above that threshold is filtered, but we want a single bbox for each object in the image.

How Non-max suppression works?

Given the set of predicted bbox and probability/score along with threshold, one needs to come up with a single bbox.

A: Set of all predicted bbox with probability

B: initially empty set

• pick the one with the highest score and push into the set B
• drop that one from A.
• calculate IoU of all others with the one picked, remove all predicted box from A whose IoU ≥ thresholds.
• repeat steps 2 and 3 until there is no more bbox left in A.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI