What's the idea of Decision Tree Classifier?
The basic intuition behind a decision tree is to map out all possible decision paths in the form of a tree. It can be used for classification and regression (Decision Tree Regression ). In this post, let's try to understand the classifier.
Suppose that we have a dataset $S$ like in the figure below (ref)
![An example of dataset $S$.](https://prod-files-secure.s3.us-west-2.amazonaws.com/70a67195-bc38-429a-9695-1ad1b42ccec8/1cbd6cb2-ef8e-4e26-8ef2-e5bec1bf4521/Untitled.jpeg)
An example of dataset $S$.
![A decision tree we want.](https://prod-files-secure.s3.us-west-2.amazonaws.com/70a67195-bc38-429a-9695-1ad1b42ccec8/9b6f0402-aba1-452f-90a2-62881d6cae59/Untitled.jpeg)
A decision tree we want.
There are many algorithms which can help us make a tree like above, in Machine Learning, we usually use:
- ID3 (Iterative Dichotomiser): uses information gain / entropy.
- CART (Classification And Regression Tree): uses Gini impurity.
Some basic concepts
![Untitled](https://prod-files-secure.s3.us-west-2.amazonaws.com/70a67195-bc38-429a-9695-1ad1b42ccec8/8a4ba745-7449-4af3-a8c4-af1603d1c756/Untitled.png)
- Splitting: It is a process of dividing a node into two or more sub-nodes.
- Pruning: When we remove sub-nodes of a decision node, this process is called pruning.
- Parent node and Child Node: A node, which is divided into sub-nodes is called parent node of sub-nodes where as sub-nodes are the child of parent node.
ID3 algorithm
- ID3 algorithm (TL;DR;)
- ID3 algorithm in detail
CART algorithm
- CART algorithm (TL;DR;)
- CART algorithm in detail
Gini Impurity or Entropy?
Some points:(ref)