Multi-resolution Image Processing and Compression

Author(s): Akula Hemanth Kumar

Originally published on Towards AI.

Making computer vision easy with Monk, low code Deep Learning tool and a unified wrapper for Computer Vision.

Table of contents

1. Multi-scale Image processing
2. Image pyramids
3. Image pyramids using OpenCV
4. Image blending using Pyramids
5. Image compression

Multi-scale Image processing

Advantages

• Scaling down for fewer storage requirements.
• Scaling image up for zooming operation.
• Scaling images and combining them is used for increasing image sharpness.

Image pyramids

• This scaling up and down can be visualized as pyramids. Thus the name pyramidal image processing.

Image pyramids using OpenCV

Lower pyramids using OpenCV

%matplotlib inline
import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread("imgs/chapter8/indoor.jpg", 0)print("Original Image shape - {}".format(img.shape))lower_reso1 = cv2.pyrDown(img)print("First Lower Pyramid Image shape - {}".format(lower_reso1.shape))f = plt.figure(figsize=(15,15))
f.add_subplot(2, 1, 1).set_title('Original Image')
plt.imshow(img, cmap="gray")
f.add_subplot(2, 1, 2).set_title('Lower level 1 Image')
plt.imshow(lower_reso1, cmap="gray")
plt.show()

Output

Original Image shape - (423, 640)
First Lower Pyramid Image shape - (212, 320)

Note: Look at the scale next to the image, not the visible shape

Higher pyramids using OpenCV

%matplotlib inline
import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread("imgs/chapter8/indoor.jpg", 0)
print("Original Image shape - {}".format(img.shape))
lower_reso1 = cv2.pyrDown(img)
print("First Lower Pyramid Image shape - {}".format(lower_reso1.shape))
restored_reso = cv2.pyrUp(lower_reso1)
print(" Restored Image shape -{}".format(restored_reso.shape))
f = plt.figure(figsize=(15,15))
f.add_subplot(2, 1, 1).set_title('Original Image')
plt.imshow(img, cmap="gray")
f.add_subplot(2, 1, 2).set_title('Restored Image')
plt.imshow(restored_reso, cmap="gray")
plt.show()

Output

Original Image shape - (423, 640) 
First Lower Pyramid Image shape - (212, 320) 
Restored Image shape - (424, 640)

Gaussian Pyramid downscaling

%matplotlib inline
import numpy as np
import cv2
from matplotlib import pyplot as plt
A = cv2.imread("imgs/chapter8/sea.jpg", 1);
height, width, channel = A.shape;

gaussian_pyramid = [];

# First image in pyramid is the orginal onegaussian_pyramid.append(A);


# Then for six times we apply pyramid down functions
for i in range(5):
 A = cv2.pyrDown(A)
 B = np.zeros((height, width, 3), dtype=np.uint8)
 B[:A.shape[0], :A.shape[1], :] = A[:, :, :]
 gaussian_pyramid.append(B)img1 = np.hstack((gaussian_pyramid[0], gaussian_pyramid[1]))
img2 = np.hstack((gaussian_pyramid[2], gaussian_pyramid[3]))
img3 = np.hstack((gaussian_pyramid[4], gaussian_pyramid[5]))

out = np.vstack((img1, img2, img3))

plt.figure(figsize=(15, 15))
plt.imshow(out[:,:,::-1])
plt.show()

Laplacian Pyramid upscaling

%matplotlib inline
import numpy as np
import cv2
from matplotlib import pyplot as plt
A = cv2.imread("imgs/chapter8/outdoor.jpg", 1)

down = cv2.pyrDown(A)
up = cv2.pyrUp(down)

print(A.shape, up.shape)
laplacian_up = cv2.subtract(A, up)

plt.figure(figsize=(15, 15))
plt.imshow(laplacian_up[:,:,::-1])
plt.show()

Output

(360, 640, 3) (360, 640, 3)

Image blending using Pyramids

Continuous Integration of images

'''
Image credits: https://github.com/opencv/opencv/tree/master/samples/data
'''%matplotlib inline
import numpy as np
import cv2
from matplotlib import pyplot as plt
A = cv2.imread("imgs/chapter8/orange.jpg", 1);
B = cv2.imread("imgs/chapter8/apple.jpg", 1);


# generate Gaussian pyramid for A
G = A.copy()
gpA = [G]
for i in range(6):
 G = cv2.pyrDown(G)
 gpA.append(G)
 

# generate Gaussian pyramid for B
G = B.copy()
gpB = [G]
for i in range(6):
 G = cv2.pyrDown(G)
 gpB.append(G)


# generate Laplacian Pyramid for A
lpA = [gpA[5]]
for i in range(5,0,-1):
 GE = cv2.pyrUp(gpA[i])
 L = cv2.subtract(gpA[i-1],GE)
 lpA.append(L)
 
# generate Laplacian Pyramid for B
lpB = [gpB[5]]
for i in range(5,0,-1):
 GE = cv2.pyrUp(gpB[i])
 L = cv2.subtract(gpB[i-1],GE)
 lpB.append(L)
 

#Now add left and right halves of images in each level
LS = []
for la,lb in list(zip(lpA,lpB)):
 rows,cols,dpt = la.shape
 ls = np.hstack((la[:,0:cols//2], lb[:,cols//2:]))
 LS.append(ls)

# now reconstruct
ls_ = LS[0]
for i in range(1,6):
 ls_ = cv2.pyrUp(ls_)
 ls_ = cv2.add(ls_, LS[i])

# image with direct connecting each half
real = np.hstack((A[:,:cols//2],B[:,cols//2:]))
#cv2.imwrite('Pyramid_blending2.jpg',ls_)
#cv2.imwrite('Direct_blending.jpg',real) 

f = plt.figure(figsize=(15,15))
f.add_subplot(2, 1, 1).set_title('Pyramidal Blending');
plt.imshow(ls_[:, :,::-1])
f.add_subplot(2, 1, 2).set_title('Direct Blending');
plt.imshow(real[:, :,::-1]);
plt.show()

Image and Video Compression

• By the end of 2020, the digital world is expected to have generated 45 Zettabytes of data. Most of which are images and videos. On average 80 % of daily internet data usage of individual results out of videos and images.

Urgent need for lossless data compression techniques

• Less data transfer
• Less data Storage

Popular image compression techniques

Lossless compression

• Run-length encoding used for BMP files.
• DEFLATE data compression used for PNG files.
• LZW compression used for GIFs

Lossy compression

• JPEG

You can find the complete jupyter notebook on Github.

If you have any questions, you can reach Abhishek and Akash. Feel free to reach out to them.

I am extremely passionate about computer vision and deep learning in general. I am an open-source contributor to Monk Libraries.

You can also see my other writings at:

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI