Scalar Multiplication
Multiplying every entry of a vector or matrix by the same scalar.
- c > 1 stretches; 0 < c < 1 shrinks; c < 0 flips.
- With vector addition, defines a vector space.
- Learning rate × gradient is scalar multiplication.
When to use: Any per-element rescaling: learning rates, attention temperatures, KL weights.
Example: 3 · [1, 2, 4] = [3, 6, 12]. Each gradient step computes 0.01 · gradient.