Source: Image by Bela Geletneky from Pixabay

Machine Learning, Editorial, Programming

Decision Trees in Machine Learning (ML) with Python Tutorial

Diving into decision trees in machine learning (ML) with Python

Last updated, January 8, 2021

Author(s): Saniya Parveez, Roberto Iriondo

Join us ↓ | Towards AI Members | The Data-driven Community

This tutorial’s code is available on Github and its full implementation as well on Google Colab.

Table of Contents

1. What is a Decision Tree?
2. Decision Tree Example
3. Building a Decision Tree
4. Node Impurity
5. Entropy
6. Gini
7. Overfitting in Decision Tree Learning
8. Pruning
9. Advantages and Disadvantages of Decision-tree-based Classification
10. Code Implementation
11. Advanced Decision Trees
12. Conclusion
13. Resources
14. References

📚 Check out our gradient descent tutorial. 📚

What is a Decision Tree?

A decision tree is a vital and popular tool for classification and prediction problems in machine learning, statistics, data mining, and machine learning [4]. It describes rules that can be interpreted by humans and applied in a knowledge system such as databases. Fundamentally, a decision tree T encodes d (a classifier or regression function) in the form of a tree structure which presents the following attributes:

• Decision node: It defines a test on a single attribute.
• Leaf node: It shows the value of the target attribute.
• Edge: It is a split of one attribute.
• Path: It is a disjunction of the test to make the final decision.