Addition
Combining two or more numbers into a total.
- Commutative and associative.
- Identity is 0; subtraction = addition of negative.
- On vectors/matrices, addition is elementwise.
When to use: Every aggregation: batch loss sums, gradient accumulation, total counts.
Example: Average loss = (sum of 32 individual losses) / 32.