A Beginners Guide to Synthetic Data

Author(s): Supreet Kaur

Originally published on Towards AI.

A Beginner’s Guide to Synthetic Data

Data for Machine Learning Model is like the heart of the human body. A model’s success depends on multiple factors, but Data is one of the critical factors that dictate success. Some companies have abundant data and no issues, but some struggle to find adequate data to build a working AI model. The alarming statistic that 80% of a Data Scientist’s time is invested in preparing data indicates the importance of “good” and “sufficient” data.

As the name suggests, “Synthetic Data” technology enables practitioners to generate data similar to actual data but customized per your requirements, volume needed, and use case. It is generated using different techniques, some of which will be discussed in this blog.

Synthetic Data fits in the use cases below:

1. Beneficial for organizations lacking a plethora of data but still wanting to build AI-driven products
2. It can be helpful in case of an imbalanced dataset. The non-dominating class data can be generated by leveraging synthetic data techniques.
3. Highly regulated industries that can’t use PII to train their model, so they generate something similar to the original data rather than using the actual data. Imagine a new team joining your organization to build a prediction model on medical image data; rather than using the actual data, which might have patient information, you decide to generate a dataset that represents that information, but at the same time, since it is not an original data, it is successfully able to mask the information.
4. Autonomous Vehicle companies have heavily relied on Synthetic Data to generate all possible edge cases to train their model. They heavily rely on simulation techniques to generate synthetic data.

Techniques to generate Synthetic Data

There are different types of techniques that can be used to generate Synthetic Data. Some are simple statistical techniques, and others are deep-learning techniques like GANs.

Statistical Methods

Data samples can be generated from a probability distribution with certain characteristic statistical features like mean, variance, skew, etc. For instance, in the case of COVID detection, one assumes that the negative samples belong to a specific statistical distribution. In contrast, the positive samples do not correspond to this data distribution. Synthetic Data can rescue in unexpected situations, such as a Pandemic, where the data does not exist. Here, we can use any existing pandemic data from public reports to generate COVID data.

Deep Learning Methods

Generative Adversarial Network (GAN): GANs are a popular method to generate synthetic data. It is an algorithm that creates fake data, i.e., very close to accurate data. There are two primary components of GANs: Discriminator and Generator. The generator is the one responsible for generating fake data, while the discriminator is the one that classifies if the generated data is close to actual data. It then provides feedback to the generator.

GANs can sometimes learn to generate only a limited set of outputs, or “modes,” rather than exploring the whole space of possible outputs. This is known as mode collapse and can result in repetitive or low-quality generated data.

An alternative approach to GANs is WGAN. The objective of a standard GAN is to minimize the Jensen-Shannon divergence between the actual data distribution and the generated distribution, while for WGANs, the goal is to minimize the Wasserstein loss function. The Wasserstein distance is a more meaningful measure of the distance between probability distributions, as it captures the amount of “work” needed to transform one distribution into the other rather than evaluating the actual output.

Open Source Technologies

1. Time Series Generator: Python package that generates time series data
2. Kubric: It is an open-source python framework launched by Google that aims to create synthetic image datasets
3. Copulas: Python library for modeling multivariate distributions and sampling from them using copula functions. Given a table of numerical data, it used copulas to learn the distribution and generate new synthetic data following the same statistical properties.
4. Pydbgen: Python package that generates a random database table based on the user’s choice of data types. This generates some standard fields like Name, Age, etc.
5. Gretel Synthetics: Leverages Recurrent Neural Networks(RNN) to generate synthetic data for structured and unstructured texts.

Limitations of Synthetic Data

1. Lack of Diversity: Synthetic data can sometimes lack the diversity and complexity of real-world data. This can result in models performing well on synthetic data but not generalizing well to real-world data.
2. Incomplete Representation: Synthetic data may not always fully capture the complexity of real-world data. For example, it may not account for rare or unexpected events that could impact the performance of the model.
3. Biases: Synthetic data may be biased if the process used to generate it is biased or if the real-world data used to train the generator is biased. This can lead to models that perpetuate existing biases or create new ones.

Companies are moving towards adapting Data Centric AI. Synthetic Data can come in handy to move towards that approach. Though it has its pros and cons, with ongoing research, it can aid breakthrough use cases and help solve cold start problems.

References:

https://analyticsindiamag.com/a-guide-to-generating-realistic-synthetic-image-datasets-with-kubric/#:~:text=Kubric%20is%20an%20open%2Dsource,functions%20of%20PyBullet%20and%20Blender.&text=Kubric%2C%20a%20scalable%20dataset%20generator,computer%2Dgenerated%20images%20and%20videos.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI