Exponent
A small superscript indicating how many times to multiply a base by itself.
- Exponents grow extremely fast.
- a^(1/2) = square root.
- Compute scaling laws use powers.
When to use: Squared errors, exponential decay, softmax, scaling-law analysis.
Example: Squared error: (y − ŷ)². GPT-3 ≈ 175 × 10⁹ params. Scaling: loss ∝ C^(−0.05).