Magnitude
The length of a vector: square root of the sum of squared entries.
- Distance between u and v equals ||u - v||.
- Magnitude 1 = unit vector.
- Gradient clipping caps ||gradient||.
When to use: Length comparisons, gradient clipping, embedding normalization.
Example: ||[3, 4]|| = 5. torch.nn.utils.clip_grad_norm_(params, max_norm=1.0) caps gradient magnitude.