GANsformers: Generate complex scenes using GANs and Transformers

Author(s): Louis Bouchard

Originally published on Towards AI.

They basically leverage transformers’ attention mechanism in the powerful StyleGAN2 architecture to make it even more powerful!

Results examples on generating bedroom scenes with its attention maps. Image from: Drew A. Hudson and C. Lawrence Zitnick, Generative Adversarial Transformers, (2021).

Last week we looked at DALL-E, OpenAI’s most recent paper.It uses a similar architecture as GPT-3 involving transformers to generate an image from text. This is a super interesting and complex task called text-to-image translation. As you can see in the video below, the results were surprisingly good compared to previous state-of-the-art techniques. This is mainly due to the use of transformers and a large amount of data.

This week we will look at a very similar task called… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI