Orthogonality
Two vectors are orthogonal when their dot product is zero (90° angle).
- In R² / R³ matches geometric perpendicular.
- Orthonormal: pairwise orthogonal AND unit norm.
- Orthonormal bases are numerically the most stable.
When to use: Stable initialization, PCA, attention head specialization, decorrelating features.
Example: [1, 0] · [0, 1] = 0 - orthogonal. Standard basis of R^n is mutually orthonormal.