Hyperparameter
A model setting chosen before training rather than learned from data.
- Parameters are learned; hyperparameters are set.
- Examples: max_depth, learning_rate, alpha, n_estimators, K.
- Tuned via cross-validation or a held-out set.
When to use: Always; a great algorithm with bad hyperparameters often loses to a mediocre one with good ones.
Example: Random Forest hyperparameters: n_estimators=500, max_depth=12, min_samples_leaf=20, max_features=sqrt. Tree weights inside are learned during training.