Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Read by thought-leaders and decision-makers around the world. Phone Number: +1-650-246-9381 Email: [email protected]
228 Park Avenue South New York, NY 10003 United States
Website: Publisher: https://towardsai.net/#publisher Diversity Policy: https://towardsai.net/about Ethics Policy: https://towardsai.net/about Masthead: https://towardsai.net/about
Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Founders: Roberto Iriondo, , Job Title: Co-founder and Advisor Works for: Towards AI, Inc. Follow Roberto: X, LinkedIn, GitHub, Google Scholar, Towards AI Profile, Medium, ML@CMU, FreeCodeCamp, Crunchbase, Bloomberg, Roberto Iriondo, Generative AI Lab, Generative AI Lab Denis Piffaretti, Job Title: Co-founder Works for: Towards AI, Inc. Louie Peters, Job Title: Co-founder Works for: Towards AI, Inc. Louis-François Bouchard, Job Title: Co-founder Works for: Towards AI, Inc. Cover:
Towards AI Cover
Logo:
Towards AI Logo
Areas Served: Worldwide Alternate Name: Towards AI, Inc. Alternate Name: Towards AI Co. Alternate Name: towards ai Alternate Name: towardsai Alternate Name: towards.ai Alternate Name: tai Alternate Name: toward ai Alternate Name: toward.ai Alternate Name: Towards AI, Inc. Alternate Name: towardsai.net Alternate Name: pub.towardsai.net
5 stars – based on 497 reviews

Frequently Used, Contextual References

TODO: Remember to copy unique IDs whenever it needs used. i.e., URL: 304b2e42315e

Resources

Unlock the full potential of AI with Building LLMs for Productionβ€”our 470+ page guide to mastering LLMs with practical projects and expert insights!

Publication

Image Processing with Fourier Transform
Computer Vision   Latest   Machine Learning

Image Processing with Fourier Transform

Last Updated on August 1, 2023 by Editorial Team

Author(s): Albert Nguyen

Originally published on Towards AI.

Image Processing is a crucial task in Computer Vision. For example, when we train a Deep Learning model with a small amount of image data, we need to synthesize new images using Image Processing methods to improve the performance.

There are many methods developed for Image Processing. This post will explore the application of 2D Fourier Transform methods to process images.

Background – Fourier Series

The Fourier series is found by the mathematician Joseph Fourier. He stated that any periodic function could be expressed as a sum of infinite sines and cosines:

More detail about the formula here

Fourier Transform is a generalization of the complex Fourier Series. In image processing, we use the discrete 2D Fourier Transform with formulas:

Image in the frequency domain

Now we have the formulas, let's see what it looks like this when applied to an image:

The image on the right side is a spectrum of Fourier Transform. It is the heat map where the color represents the sine waves' amplitude, and position is their frequency. But at this point, we may get confused, like, "Hey," what does this tell me about the image?"

What we are doing with the 2D Fourier Transform is treating the image as a function of pixel position x and y. Imagine the function f(x, y) along axis (0,y):

So when the color jumps from "black (0)" to "white (255)", we say the color changes quickly, which means that a high amplitude and frequency sine wave contribute to that jump. So the low-frequency sine waves are responsible for the image's overall color, while high-frequency waves are responsible for how color changes.

To be honest, I can't find a better term for the responsibility of the high-frequency wave. However, the example below demonstrates my idea.

The cat image when we remove low frequency waves.

So, when we remove the low-frequency terms, we get an edge detection of the cat. On the opposite side, if we only keep the low-frequency waves, we will get a blurred image:

This is so cool when we all need to delete some terms, and we can do β€˜blurring’ and β€˜edge detection.’

Well, this is nothing surprising. We can do all this with openCV. So, what else can Fourier Transform do?

Fourier Transform and Convolution

The beauty of the Fourier Transform is we can do convolution on images by just multiplication on its frequency domain. The convolution measures the total product in the overlapping regions of 2 functions. In Deep Learning, we often know about it as a convolution layer. While mathematically, it will look like this:

If we plug the Fourier Transform formula into this operation for f and g, we will get:

This is so nice. With Fourier transform, we can change from computing a convolution to a multiplication. In the case of 2D Discrete Fourier Transform, this helps us reduce the time complexity by a lot.

Source: https://www.robots.ox.ac.uk/~az/lectures/ia/lect2.pdf

Conclusion

Fourier Transform is a powerful tool and is widely used in many applications. In this blog, we have explored some usage of the FT in image processing. Fourier Transform is also used in some other applications in Deep Learning, which I find interesting and listed below:

  1. Domain Adaption for Semantig Segmentation

2. Resolution-robust Large Mask Inpainting with Fourier Convolutions

3. etc.

Besides the frequency representation, the Fourier Transform also produces the phase representation of the image. This contains the spatial information inside the image. However, the plot for the phases domain is less informative, and it also makes functions that affect the spatial information become more complex.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming aΒ sponsor.

Published via Towards AI

Feedback ↓