Decision Tree
Recursively splits the feature space into rectangles, predicting a constant within each region.
- Splits chosen to maximize purity (Gini or entropy).
- Predicts majority class or mean target in each leaf.
- Handles mixed types, missing values, and interactions natively.
When to use: When you want interpretability or as the base learner inside forests and boosting.
Example: Loan approval tree: split on income > 50k, then credit_score > 680, then debt_ratio < 0.4. Each leaf stores an approval probability.