Logistic Regression: A Probabilistic Perspective

Author(s): Reza Bagheri

Originally published on Towards AI.

The mathematical relationship between logistic regression, the logistic distribution, and the sigmoid function

This member-only story is on us. Upgrade to access all of Medium.

Logistic regression is a simple but powerful supervised machine-learning algorithm. It is widely used for binary classification problems where the target only has two categories. Logistic regression uses the sigmoid (aka logit function) to predict the probability that a data point belongs to each category.

Logistic regression is discussed in many data science or machine learning textbooks. However, there are a few questions that usually remain unanswered. Why do we use the sigmoid function to calculate the probability? When calculating a probability, we need to have a probability distribution. Does the sigmoid function (or logistic regression) assume that a certain probability distribution generates the dataset? In linear regression, the main assumption is that the data points contain a noise term with a normal distribution. Is there a similar assumption for logistic regression? In this article, we will answer these questions.

All the images in this article were created by the author.

Logistic regression

Logistic regression is used for binary classification problems. Suppose that you have a dataset with the features x₁, x₂,…x_n and the target y that takes the values -1 and 1. The following equation is the backbone of the logistic regression… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI