Linear Combination
A sum of vectors each multiplied by a scalar coefficient.
- Set of all linear combinations = span.
- Every NN layer builds features as linear combinations + nonlinearity.
- PCA represents data as linear combinations of components.
When to use: Anywhere features, predictions, or directions are formed by weighted sums.
Example: 2·[1, 0] + 3·[0, 1] = [2, 3]. Multi-head attention output is a linear combination of value vectors.