Exponentials and Logarithms

Growth that feeds on itself#

Linear growth adds a fixed amount per step; exponential growth multiplies by a fixed factor per step. Money at compound interest, populations, radioactive decay (backwards), epidemics — all are exponential, and all eventually dwarf any polynomial.

\begin{aligned}y = a \cdot b^x: \\ a = \text{starting value}, \\ b = \text{growth factor per unit of } x.\end{aligned}

Among all possible bases one is nature’s favorite: e = 2.71828\dots, the base for which the curve’s slope equals its height at every point. It arises as the limit of ever-finer compounding: e = \lim_{n \to \infty}\left(1 + \frac{1}{n}\right)^n.

The logarithm: exponentiation’s answer key#

\log_b y answers one question: to what power must I raise b to get y?

\begin{aligned}\log_b y = x \iff b^x = y. \\ \log_2 8 = 3, \\ \log_{10} 1000 = 3, \\ \ln e^5 = 5.\end{aligned}

The exponential and the logarithm undo each other, so their graphs are mirror images across y = x:

Because logs turn exponents into multipliers, they turn hard operations into easy ones — this is precisely why they were invented (Napier, 1614, to save astronomers from arithmetic):

\begin{aligned}\log(xy) = \log x + \log y, \\ \log\frac{x}{y} = \log x - \log y, \\ \log(x^n) = n\log x.\end{aligned}

And the change of base formula, \log_b x = \dfrac{\ln x}{\ln b}, means one log button on a calculator suffices for all bases.

Key idea. A logarithm measures orders of magnitude. The Richter scale, pH, decibels, and octaves are all logarithmic, because human questions about vastly-ranging quantities are really questions about exponents.

Solving exponential equations is a one-move game — take logarithms — but let us play it slowly, twice.

Friendly numbers. \begin{aligned}5 \cdot 2^x &= 160\end{aligned}

the equation

\begin{aligned}2^x &= 32\end{aligned}

divide both sides by 5 — isolate the exponential first

\begin{aligned}2^x &= 2^5 \\ &\quad \text{recognize } 32 = 2^5 \\ x &= 5\end{aligned}

same base, so exponents must match.

Unfriendly numbers — when you cannot spot the power, the logarithm’s rule \log(a^x) = x\log a pulls the unknown down out of the exponent, which is the entire reason logs help: \begin{aligned}3^x &= 20\end{aligned}

no obvious power of 3

\begin{aligned}\ln(3^x) &= \ln 20 \\ &\quad \text{take } \ln \text{ of both sides (any log works)} \\ x \ln 3 &= \ln 20\end{aligned}

the power rule: the exponent comes down as a multiplier

\begin{aligned}x &= \frac{\ln 20}{\ln 3} \approx \frac{2.996}{1.099} \approx 2.73. \\ &\quad \text{divide — } x \text{ is now in ordinary algebra}\end{aligned} Check: 3^{2.73} \approx 20.0. ✓

Worked example (doubling time). An investment grows at 7% per year. Doubling requires 1.07^n = 2, so n = \frac{\ln 2}{\ln 1.07} \approx 10.2 years. The shortcut “72 \div \text{rate}” gives 72/7 \approx 10.3 — the rule of 72, and Exercise 3 shows where the 72 comes from.

In the wild. Logarithms are the working coordinates of computing and ML: algorithmic complexity (O(\log n) binary search, O(n\log n) sorting), log-scaled training-loss curves, log-probabilities (multiplying thousands of probabilities underflows to zero; adding their logs is stable), decibels in audio, and the log-loss/cross-entropy of Chapter 23. When numbers span many orders of magnitude, professionals stop plotting x and start plotting \log x.

If you keep one thing from this chapter: A logarithm is a question about exponents. It turns multiplication into addition and unreadable vastness into readable slopes — that is why it was invented and why it never left.