Overfitting
A model memorizes training quirks and fails to generalize, low train error but high test error.
- Signature: training near-perfect, validation much worse.
- The gap widens as model complexity grows.
- Fixes: regularization, more data, simpler models, early stopping.
When to use: Diagnose first whenever validation error is much worse than training error.
Example: A depth-30 decision tree hits 99% training accuracy and only 62% test accuracy. Capping depth at 6 gives 84% train and 80% test, a far healthier profile.