What We Called “Deepfakes” Are No Longer Fakes: How To Use the Technology for Good

Author(s): Ivan Altsybieiev

Originally published on Towards AI.

Artificial Intelligence, Technology

How people use synthetic media to facilitate personalized communication

We have been working on machine learning for nine years. Today our face swap app Reface is used by millions of people every day. Despite our app's popularity and security, face-swap technology is still wrongly associated with“deepfakes” that spread false information.

Cheese is not chalk

The term “deepfakes” originated from a Reddit user, who in December 2017 applied a machine-learning algorithm to produce celebrity pornographic fakes. Media soon began to use “deepfakes” as a collective term for any synthesized content. Mistakenly, people called deepfakes, including video clips created with no deep-learning technology or ones not intended to deceive viewers.

Face-swapping technology, which we developed for Reface, is a type of synthetic media, which operates with a universal neural network that changes facial features with just one photo in seconds. This state-of-the-art tech is a massive breakthrough in personalized communication. It stands alongside Instagram selfie masks or IOS emoji avatars and is accessible to everyone who owns a smartphone.

Below are examples of AI-generated content used for good in different areas over the last two years.

● The Dalí Museum created a video-installation using AI, which looks like it stars the real Salvador Dalí and helps visitors connect to his art in a new way. Dalí Lives was made using a machine-learning algorithm that studied its subject across 6,000 archival frames of video.

● Lip-syncing technology can adapt content for foreign audiences into different languages. This was the case of Synthesia.io and their video with David Beckham speaking nine different languages to raise malaria awareness.

● Fascinating Respeecher’s voice-cloning technology replicates voices and produces synthetic speeches that are indistinguishable from the originals. It brings the film dubbing field to a whole new level, and we hear the actors' original voices in other languages.

Using the “deepfake” label to describe the examples above is a naming error. “Synthetic media” is a better term to denote AI-generated audiovisual content.

Customer use of face-swap technology

We gathered feedback from our users and potential customers to get insights into their user experience and test our hypothesis. The survey revealed that 73% of respondents use face-swap technology for fun. Another 22% are looking for a tool to boost their product, organization, or marketing performance. Some 5% of responses surprised even us, e.g., improving mental health, making marriage proposals.

Below are some of the most remarkable ways people use face-swap technology, which we discovered during our cust-dev research.

1. To improve mental healthcare and body insecurities

Although the face-swap technology is relatively new and needs further research, it’s already promising for healthcare. Reface users have acknowledged our tech has a great potential to outline new approaches in personalized treatments for patients with autism spectrum disorder and social exclusion.

“Face-swapping helps them immerse in a learning environment with empowered versions of themselves and shows how they can genuinely interact with others and form the relationships,” said one respondent.

Our research has shown this technology's relevancy for people with body dysmorphic disorder, trans and non-binary people. Refacing can improve their experience by instantly helping them visualize themselves in the body or gender they feel on the inside.

2. Anonymization

Reface technology can swap faces and bodies on actual videos and images with synthetically altered versions of people. The system works with vectors and changes face to a similar one — of the same sex and race. As a result, the image looks generic and can be used for commercial purposes without privacy rights restrictions. This can significantly facilitate the routine of photographers and videographers.

3. To create a complete and immersive gaming experience

One of the most potent insights we got from our research is that refacing is a new way to increase the immersive gaming experience. The technology improves the quality of characters’ facial expressions and mimics and advances similarity with the prototype.

As companies will incorporate face-swap technologies, the gamers’ experience will become more personalized, allowing them to create characters based on their appearance.

“We are working on a sports game, so we wanted the user model to look just like the player. The goal is simply to increase immersion as much as possible. Virtual reality puts you in the game and we wanted the player to be able to share that experience on social media platforms,” said our respondent who works in the gaming industry.

4. To facilitate personalized communication

Why settle for a boring happy birthday card if you can send your BFF a surreal record of them dancing in the video of Cardi B, Madonna, or Taylor Swift?

Want to watch a new movie trailer with your face in a leading role? It will take just 15 minutes and it’s no longer rocket science.

Personalization may be the new black, but it’s also a new competitive battleground. Empowered by AI, people are looking for new ways of self-expression and inspiration. And face-swapping tech has brought personalized content to a new level. Be it creating GIFs to spam your friends, editing video clips to congratulate someone, working on educational content or customized marketing for your clients, or even honoring one’s family archive by swapping faces of family members.

One of the survey participants shared that Reface helped him implement his proposal idea. He recreated a F.R.I.E.N.D.S reference video and swapped Chandler’s and Monica’s faces with his and his girlfriend’s.

5. A new, vital tool for creative industries

Among other fields where face-swap technology has considerable potentials are cinema and visual arts.

Productions can use face-swapping tools to try various actors for the role, test different makeup looks and styles, create artificial actors, conduct virtual casting probes, fix production mess-ups, change or substitute actors, and work with various locations.

The pandemic is accelerating the ongoing transformation in movie production, distribution, and consumption. However, face-swap has already been proven an effective solution within the COVID-19 restrictions.

For example, Ukrainian brand KSENIASCHNAIDER has launched a Spring-Summer 2021 Runway Video using Reface technology to alter the models’ appearance. This way, using only three models on the catwalk and a face-swapping app, the brand created more than 20 unique looks with minimum resources spent. Technology makes the entire casting process not just safer but shorter and less costly.

Synthetic media in general and Reface technology specifically provide various opportunities. It augments Youtube, TV and filmmaking, marketing, gaming, social services, and healthcare industries.

It’s important to use correct terms and feel the difference between technology development and its misuse. Like any new technology, the creation of AI-generated content requires regulations, restrictions, and supervision. To achieve this objective, we have adopted a content moderation policy and designed deepfake detection tools. We are committed to ensuring our technology is used for the right purposes, is not offensive, and does not violate anyone’s rights.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI