Dot Product
A scalar produced by elementwise-multiplying two equal-length vectors and summing.
- Positive: same direction; zero: perpendicular; negative: opposite.
- Atomic op of every neuron and attention head.
- Normalized → cosine similarity.
When to use: Measuring alignment between two vectors; foundation of every linear layer and attention.
Example: [1, 2, 3] · [4, 5, 6] = 32. Two unit vectors with dot product 0.95 are nearly aligned.