Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Read by thought-leaders and decision-makers around the world. Phone Number: +1-650-246-9381 Email: [email protected]
228 Park Avenue South New York, NY 10003 United States
Website: Publisher: https://towardsai.net/#publisher Diversity Policy: https://towardsai.net/about Ethics Policy: https://towardsai.net/about Masthead: https://towardsai.net/about
Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Founders: Roberto Iriondo, , Job Title: Co-founder and Advisor Works for: Towards AI, Inc. Follow Roberto: X, LinkedIn, GitHub, Google Scholar, Towards AI Profile, Medium, ML@CMU, FreeCodeCamp, Crunchbase, Bloomberg, Roberto Iriondo, Generative AI Lab, Generative AI Lab Denis Piffaretti, Job Title: Co-founder Works for: Towards AI, Inc. Louie Peters, Job Title: Co-founder Works for: Towards AI, Inc. Louis-François Bouchard, Job Title: Co-founder Works for: Towards AI, Inc. Cover:
Towards AI Cover
Logo:
Towards AI Logo
Areas Served: Worldwide Alternate Name: Towards AI, Inc. Alternate Name: Towards AI Co. Alternate Name: towards ai Alternate Name: towardsai Alternate Name: towards.ai Alternate Name: tai Alternate Name: toward ai Alternate Name: toward.ai Alternate Name: Towards AI, Inc. Alternate Name: towardsai.net Alternate Name: pub.towardsai.net
5 stars – based on 497 reviews

Frequently Used, Contextual References

TODO: Remember to copy unique IDs whenever it needs used. i.e., URL: 304b2e42315e

Resources

Unlock the full potential of AI with Building LLMs for Productionβ€”our 470+ page guide to mastering LLMs with practical projects and expert insights!

Publication

Why not Linear Regression?
Latest   Machine Learning

Why not Linear Regression?

Last Updated on July 26, 2023 by Editorial Team

Author(s): Akash Dawari

Originally published on Towards AI.

All About Logistic Regression

In this article we will understand Logistic Regression and try to answer the following questions:

  • What is Logistic Regression?
  • Why not Linear Regression?
  • Why Logistic Regression?
  • When Logistic Regression?
  • How Logistic Regression?

What is Logistic Regression?

Logistic Regression is a Supervised Machine Learning algorithm that is used in classification problems where we have to distinguish the dependent variable between two or more categories or classes by using the independent variables.

For example:

Iris Flower Dataset

Consider the above table where the dependent variable is β€˜Species’ and the independent variable is β€˜petal width’. Species show two categories that are β€˜0’ denotes Setosa iris flower and β€˜1’ denotes Versicolor iris flower.

Logistic Regression will give the probability of the respective category using the petal width of the flower.

There are mainly two reasons:

  1. Linear regression deal with continuous or quantitative values whereas in classification problems we deal with discrete or probabilistic values.
  2. If we somehow fit the linear regression line into the data set as follows:
Graph of Iris Dataset with linear regression

If we take the threshold of 0.5, in this case, the linear line seems to do a good job but if we introduce some outliers in the dataset.

Graph of Iris Dataset(with outliers) with linear regression

You can see how much the line slope fluctuates from its original position. So we cannot afford such a model which gives different answers when it is introduced with new data points. Another small point to be noticed is that the range of the outcome is not in the range that we want, which is predicting probability outside the range of 0 to 1.

To know more about Linear Regression click on the below link:

All About Linear Regression

In this article, we will explore Linear Regression and try to answer the following question:

medium.com

Why Logistic Regression?

  • As we discuss in the above question, linear regression fluctuates when new data points are introduced. Logistic regression solves this problem by introducing an additional function to bend the best fit straight line into a curve shown below.
Graph of Iris Dataset with logistic regression
  • Logistic regression is easy to understand, implement, and very efficient to train.
  • Logistic regression works well on small datasets and is very fast at classifying unknown records.
  • The core concept of logistic regression is further used in Artificial Neural networks.

When Logistic Regression?

When the following requirements are satisfied for logistic regression:

  • The dependent variable must be categorical data either be the binary class of data or the ordinal class of data.
  • Observation of the independent variables must not be repetitive or matched, cause logistic regression is sensitive to overfitting.
  • Multicollinearity should not be there among the independent variables
  • Logistic regression typically required large sample size.

NOTE: Above points are also referred to as the β€œAssumptions for logistic regression”.

How Logistic Regression?

There are two approaches to understanding logistic regression.

  1. Probabilistic Intuition
  2. Geometric Intuition

In this article, we will go for Probabilistic Intuition as Geometric Intuition will again cover in SVM (Support Vector Machine).

As we discuss above logistic regression is using a function to squash or bend the linear line that function can be any mathematical function like tan, Sigmoid, ReLu, etc

Sigmoidal Function

Logistic Regression mostly use the sigmoid function because-

  1. sigmoid function return outcome in the range of 0 to 1 which is well suited for logistic regression as we are predicting the probability of the outcome.
  2. Sigmoid Function derivative is easy to calculate as compared to other function which is used during gradient descent.
Sigmoid Function

Let’s see some relation between the probability of outcome and the linear line equation to better understand the relation between the linear regression and logistic regression:

Logit Function in Logistic Regression

Now, as most supervised machine learning Algorithms, logistic regression also has three main steps: Hypothesis, cost function, an optimization technique.

Hypothesis: In this, we assume some kind of mathematical relationship between dependent and independent variables. In logistic regression, we are using the Sigmoid function to establish the relationship.

Cost function: this function is used to find out the error of our predicted value. In logistic regression, we are using the log loss function.

Log Loss Function

Optimization Technique: In this step, we are trying to reduce the error using some mathematical techniques. In logistic regression, we will use Gradient Descent to do that.

Gradient Descent

where derivative of the log loss function is:

Coding Example of Logistic Regression Using Python from scratch:

Step 1: Preparing the dataset

Step 2: Creating a function that performs Gradient Descent and returns the value of weights associated with each independent variable.

Step 4: Creating a function that predicts the outcome using the weights.

Step 5: Call the functions and predict the outcomes.

How do we know the score or accuracy of the prediction?

In classification problems, we use the confusion matrix, accuracy score, precision, recall, or F1 score to measure the accuracy of the model.

Confusion Matrix:

Confusion Matrix

where,
True positive(TP): A result that was predicted as positive by the classification model and also is positive
True Negative(TN): A result that was predicted as negative by the classification model and also is negative
False Positive(FP): A result that was predicted as positive by the classification model but actually is negative
False Negative(FN): A result that was predicted as negative by the classification model but actually is positive.
The Credibility of the model is based on how many correct predictions did the model do.

Accuracy score: The total number of correct classifications divided by the total number of classifications.

Precision: It is a measure of amongst all the positive predictions, how many of them were actually positive

Recall: It is a measure of: from the total number of positive results how many positives were correctly predicted by the model.

F1 Score: It is defined as the harmonic mean of Precision and Recall.

link of the notebook used in this article:

Articles_Blogs_Content/All_About_Logistic_Regression.ipynb at main ·…

This repository contains jupyter notebooks regarding to the Articles published in blogs. …

github.com

Like and Share if you find this article helpful. Also, follow me on medium for more content related to Machine Learning and Deep Learning.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming aΒ sponsor.

Published via Towards AI

Feedback ↓