ML Algorithms Glossary

Foundational to advanced terms in classical machine learning algorithms

Machine Learning - Algorithms that learn patterns from data instead of being explicitly programmed.
Supervised Learning - Learning a mapping from inputs to known labels using labeled training examples.
Unsupervised Learning - Finding structure or groupings in unlabeled data, with no target values to predict.
Reinforcement Learning - An agent learns by trial and error, receiving rewards or penalties for its actions.
Classification - A supervised task that assigns each input to one of a discrete set of categories.
Regression - A supervised task that predicts a continuous numeric value rather than a category.
Clustering - An unsupervised task that groups similar examples together based on distance or density.
Feature - An individual measurable input variable used by a model to make predictions.
Label - The known answer for each training example, what a supervised model is trying to predict.
Training Set - The portion of labeled data used to fit the model parameters.
Test Set - A held-out portion of labeled data touched once, at the end, to estimate generalization.
Overfitting - A model memorizes training quirks and fails to generalize, low train error but high test error.
Underfitting - A model too simple to capture real patterns, performs poorly on both train and test.
Linear Regression - Predicts a continuous output as a weighted sum of features, fit by minimizing squared errors.
Logistic Regression - A linear model for classification that maps a linear score through a sigmoid to a probability.
Decision Tree - Recursively splits the feature space into rectangles, predicting a constant within each region.
Random Forest - An ensemble that averages many decorrelated decision trees trained on bootstrap samples.
K-Means - Partitions data into K clusters by alternating assign-to-nearest and recompute-centroids.
Neural Network - Layers of weighted sums and non-linear activations, trained by backpropagation.
Accuracy - The fraction of predictions that match the true label.
Cross-Validation - Train and test on multiple splits of the data to get a more reliable performance estimate.
Hyperparameter - A model setting chosen before training rather than learned from data.