Description
part a
Console Output:
Visualized Learned Decision Tree:
I coded with the given frame and designed my own ID3 Algorithm. In the following part b-e, I
will import these functions directly from decision_tree.py.
For part b-e, you should execute Assignment2b-e.py instead.
part b
Console Output:
Plots implemented by matplotlib:
Max Depth 1 3 5
AVG Training Error 15.73% 17.57% 10.00%
AVG Testing Error 24.24% 29.17% 13.61%
From these three plots can we see, it is wise to choose the maximum depth with 3. Because a
decision tree with depth more than 3 is often suffered from overfitting.
part c
Console Output:
Visualized Learned Decision Tree:
Max Depth = 1
Max Depth = 3
Max Depth = 5
part d
Console Output:
Visualized Learned Decision Tree:
Max Depth = 1
Max Depth = 3
Max Depth = 5
The result I get from part d is slightly different from part c because of the way that I split the
nodes. But the overall shapes are same.
part e
ID3 on the left (top if too large) and Scikit-learn on the right (below).
Console Output:
Visualized Learned Decision Tree:
Max Depth = 1
Max Depth = 3
Max Depth = 5
At the last part, I use the Glass Identification Data Set from UCI. I did not want to make y which
had 7 different values into 0 and 1. I think that way is illogical. So I optimized a little with my code
in decision_tree.py and made it work by only transform xi’s values into 0 and 1.
I did not find a way to make Scikit-learn use equal to split. Hence, the learned decision trees
obtained by ID3 have some differences from the ones by Scikit-learn. However, if you observe
the confusion matrix, you would find the matrixes are similar correspondingly. I think this proved
the rationality of decision tree learning algorithm.
