Decision trees are literally easy
That is the fifth post in my scikit-learn tutorial series. In case you didn’t catch them, I strongly recommend my first 4 posts; it’ll be way easier to follow along.
Sklearn tutorial
This module introduces decision trees. As we’ll see, decision trees are a variety of supervised learning algorithm that works by recursively splitting the information based on threshold/feature couples, making a nested structure like a tree. The leaves of the tree represent the prediction of the model.
Personal Disclaimer: After I first began the scikit-learn tutorial, I had never encountered decision trees. At best, I’d read that term here and there but had absolutely no idea what it meant. So I used to be sort of inquisitive about what decision trees were and afraid it might be more complex and harder to grasp than other models I already knew, like linear regression models and support vector machines. Nevertheless it seems, decision trees are literally way simpler and easier to grasp! They usually are powerful!
We’ll discover how decision trees work first using a quite simple example of a regression problem with a 1d dataset and the MSE loss function, after which a 2D dataset for classification with the Gini and Entropy impurity functions. The thought is to grasp the concept of how decision trees grow, and what are the differences between a regression and a classification. It’s then easy to extrapolate the way in which they work to higher dimension problems. Finally we’ll see some hyperparameters decision trees expose.
As a way to understand and grasp the general logic behind decision trees, we’ll use a straightforward example of 1D regression, using DecisionTreeRegressor.
%matplotlib qt
import numpy as np
import matplotlib.pyplot as plt
from sklearn.tree import DecisionTreeRegressor, plot_treenp.random.seed(42)
X = np.sort(5 * np.random.rand(80, 1), axis=0)
y = np.sin(X).ravel() + np.random.normal(0, 0.1…