Exploring LaMa: Resolution-robust Large Mask Inpainting with Fourier Convolutions: A Brief Overview

Author(s): Vincent Liu

Originally published on Towards AI.

Figure 1. Inpainting examples. Source: original paper¹

Image inpainting is a computer vision technique to reconstruct the damaged or masked region based on the surrounding context in the image. One may come across LaMa, a GAN-based network published in 2022. It is known for its lightweight architecture and is specifically designed to improve large-mask cases.

What’s the problem with large masks in image inpainting?

Figure 2. Masked areas of different sizes. Source: image by author.

Image inpainting models redraw the missing part based on the surrounding patches. The larger mask makes the reconstruction more challenging owing to the greater information to be restored and also the lesser context to rely on (larger mask -> Smaller background). In Figure 2, we have 4 images with various sizes of masked areas. Without considering the complexity of the background, it can be observed that the challenges scale with the size of the rectangular masks.

LaMa specializes in restoring the large masked area with its innovative structure and loss functions. If you are curious about the idea behind it, let’s move on to the next section.

Figure 3. Network of LaMa. Source: original paper¹

Let’s start with the network architecture. LaMa is a GAN-based work consisting of a generator and a… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI