ANPR with YOLOV8

Author(s): Skander Menzli

Originally published on Towards AI.

Introduction:

YOLO V8 is the latest model developed by the Ultralytics team. It’s a state-of-the-art YOLO model that transcends its predecessors in terms of both accuracy and efficiency.

It’s easy to use and accessible from the command line or via the Python package. It offers out-of-the-box support for object detection, classification and segmentation tasks. It recently added native support for object tracking as well so we won’t have to deal with cloning repos of tracking algorithms.

In this article, I will go through the steps of utilizing YOLOV8 to build an automatic number plate recognition(ANPR) tool. So let’s get started.

Tracking vehicles:

As we mentioned earlier, YOLOV8 has native tracking, so this step is pretty straightforward. First, install ultralytics package

pip install ultralytics

Then, we have to read the video frames with open cv and apply the model track method with the persist argument set to True to ensure the ids persist through the next frame. The model returns coordinates to draw a bounding box plus the id, label, and score

import cv2
from ultralytics import YOLO

model = YOLO('yolov8n.pt')
cap = cv2.VideoCapture("test_vids/vid1.mp4")

ret = True
while ret:
 # Read a frame from the camera
 ret, frame = cap.read()
 if ret and frame_nbr % 10 == 0 :

 results = model.track(frame,persist=True)
 for result in results[0].boxes.data.tolist():
 x1, y1, x2, y2, id, score,label = result
 #check if threshold met and object is a car
 if score > 0.5 and label==2:
 cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 4)
 text_x = int(x1)
 text_y = int(y1) - 10
 cv2.putText(frame, str(id), (text_x, text_y),
 cv2.FONT_HERSHEY_SIMPLEX, 0.8, (0, 255, 0), 2)
 cropped_img = frame[int(y1):int(y2), int(x1):int(x2)]

here is the result on one frame:

The bounding boxes coordinates are then used to crop each car in the frame into an image

Number Plate Recognition:

Now that we have our cars, we need to detect license plates, for that, we need to train the Yolo model. For that, I used the following Kaggle dataset.

However the labels in this dataset are in PASCAL VOC XML :

<annotation>
 <folder>images</folder>
 <filename>Cars105.png</filename>
 <size>
 <width>400</width>
 <height>240</height>
 <depth>3</depth>
 </size>
 <segmented>0</segmented>
 <object>
 <name>licence</name>
 <pose>Unspecified</pose>
 <truncated>0</truncated>
 <occluded>0</occluded>
 <difficult>0</difficult>
 <bndbox>
 <xmin>152</xmin>
 <ymin>147</ymin>
 <xmax>206</xmax>
 <ymax>159</ymax>
 </bndbox>
 </object>
</annotation>

YOLO needs the annotations of each image in a file with the following format: label, x center, y center, width, height

This code handles that transformation of our data:

def xml_to_yolo(bbox, w, h):
 # xmin, ymin, xmax, ymax
 x_center = ((bbox[2] + bbox[0]) / 2) / w
 y_center = ((bbox[3] + bbox[1]) / 2) / h
 width = (bbox[2] - bbox[0]) / w
 height = (bbox[3] - bbox[1]) / h
 return [x_center, y_center, width, height]

def convert_dataset():
 for filename in os.listdir("annotations"):

 tree = ET.parse(f"annotations/{filename}")
 root = tree.getroot()
 name = root.find("filename").text.replace(".png", "")
 width = int(root.find("size").find("width").text)
 height = int(root.find("size").find("height").text)

 for obj in root.findall('object'):
 box = []
 for x in obj.find("bndbox"):
 box.append(int(x.text))

 yolo_box = xml_to_yolo(box, width, height)
 line = f"0 {yolo_box[0]} {yolo_box[1]} {yolo_box[2]} {yolo_box[3]}"

 with open(f"train/labels/{name}.txt", "a") as file:
 # Write a line to the file
 file.write(f"{line}\n")

now, all that’s left is to set up our config yaml with paths to the train and validation data folders, then train the model note( the folder names inside the train and validation folders should be labels and images). Then, we pass it as an argument to our model instance and start training

path: C:/Users/msi/PycharmProjects/ANPR_Yolov8
train: train
val: val

# Classes
names:
 0: license plate

model = YOLO('yolov8n.yaml')
result = model.train(data="config.yaml",device="0",epochs=100,verbose=True,plots=True,save=True)

Now that we have our license plate model, we simply have to load it and use it on the cropped car images from the video, we apply grayscale on the crop of the license plate and use easy_ocr to read its content

cropped_img = frame[int(y1):int(y2), int(x1):int(x2)]
plates = lp_detector(cropped_img)
for plate in plates[0].boxes.data.tolist():
 if score > 0.6:
 x1, y1, x2, y2, score, _ = plate
 cv2.rectangle(cropped_img, (int(x1), int(y1)), (int(x2), int(y2)), (255, 0, 0), 2)
 lp_crop = cropped_img[int(y1):int(y2), int(x1):int(x2)]
 lp_crop_gray = cv2.cvtColor(lp_crop, cv2.COLOR_BGR2GRAY)
 ocr_res = reader.readtext(lp_crop_gray)
 if not ocr_res:
 print("No plate detected")
 else:
 entry = {'id': id, 'number': ocr_res[0][1], 'score': ocr_res[0][2]}
 update_csv(entry)
 out.write(frame)

 cv2.imshow('frame', frame)
 frame_nbr += 1

the update_csv function will write the car id and license plate number into a CSV file. And that’s the ANPR pipeline with yolov8

Conclusion:

As we saw, YOLOV8 streamlines the process of building an ANPR pipeline as it offers native tracking and object detection.

this repository contains the full project where I built an ANPR app with streamlit:

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI