In-depth Handling/Imputation Techniques of Missing Values in Feature Transformation

Author(s): Amit Chauhan

Originally published on Towards AI.

Data imputation for machine learning and data science project
Photo by Emile Perron on Unsplash

As we know, machine learning algorithms are not very good with missing data. Part of feature engineering is transforming the missing data feature into a reliable feature by removing missing data rows or imputations.

It is a process to assign or substitute the missing row/column value with the desired/calculated value.

We need to remove/impute all missing data rows or columns before feeding to the machine learning model. The following methods can be used to transform missing value features.

Deleting: The easiest technique is to remove the row that has a missing value. This technique can also be called complete case analysis (CCA). This method is useful if the missing value percentage is less than 5% in a feature otherwise we can see a data loss issue.Imputation: This type of technique is to fill the missing data with some calculated value. The Imputation is divided into two methods based on univariate and multi-variate features.Uni-variate: In the case of numerical feature, the missing data can be replaced by mean/median/random value. In the case of the categoryfeature, the missing value can be replaced by mode or as a missing string. The sklearn library gives a library named Simple Imputer class that… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI