Calculus I: Limits and Derivatives

The limit: mathematics learns to whisper#

Calculus is built on one subtle move: asking not what a function is at a point, but what it is approaching. (Newton and Leibniz invented it independently in the 1660s–70s — Newton for planetary motion, Leibniz for the crisp \frac{dy}{dx} notation we still use — and their supporters feuded for decades over priority. The mathematics, indifferent to the quarrel, worked for both.) We write \lim_{x \to a} f(x) = L to mean: f(x) gets as close as anyone demands to L, whenever x is close enough to a. The function needn’t even be defined at a — limits are about the journey, not the destination.

The most important limit in this chapter is one that looks like nonsense at first: \lim_{h \to 0} \frac{f(a+h) - f(a)}{h} — a fraction whose top and bottom both go to 0. The entire differential calculus lives inside that \frac{0}{0}.

The derivative: an instant’s rate of change#

The average slope of f between a and a+h is \frac{f(a+h)-f(a)}{h} — the slope of a secant line. Slide the second point home, and the secants tilt into the tangent:

f'(a) = \lim_{h \to 0} \frac{f(a+h) - f(a)}{h}.

Before any rules, one derivative computed from the definition, every step shown — this is the computation the rules were invented to spare us, and doing it once is what makes the rules feel earned. Take f(x) = x^2:

\begin{aligned}\frac{f(x+h) - f(x)}{h} &= \frac{(x+h)^2 - x^2}{h}\end{aligned}

substitute into the definition

\begin{aligned}&= \frac{x^2 + 2xh + h^2 - x^2}{h} \\ &\quad \text{expand } (x+h)^2 \\ &= \frac{2xh + h^2}{h} \\ &\quad \text{the } x^2 \text{ terms cancel — this always happens} \\ &= \frac{h(2x + h)}{h} = 2x + h \\ &\quad \text{factor out } h \text{ and cancel it — legal because } h \ne 0 \\ &\xrightarrow{\;h \to 0\;} 2x. \\ &\quad \text{now let } h \text{ vanish}\end{aligned}

So \frac{d}{dx}x^2 = 2x. Notice the drama’s structure: the raw quotient is \frac{0}{0}-shaped, but algebra cancels the offending h first, and only then does h \to 0 — the limit tiptoes up to the cliff edge without stepping off. Every derivative rule in the table below was proved by exactly this dance.

The derivative f'(a) is the slope of the graph at a — equivalently, the instantaneous rate of change: velocity from position, marginal cost from cost, growth rate from population.

Computed once from the definition, the rules then do all future work:

The chain rule is the workhorse: rates multiply along a chain of dependence. If y changes 3 times as fast as u, and u changes 2 times as fast as x, then y changes 6 times as fast as x. In use, the ritual has four steps — name the layers, differentiate each layer, multiply, substitute back:

Differentiate y = \sin(x^2).

\begin{aligned}&\text{1. Name the layers: outer } y = \sin u, \text{ inner } u = x^2.\end{aligned}

“sin of something”

\begin{aligned}&\text{2. Differentiate each: } \frac{dy}{du} = \cos u, \qquad \frac{du}{dx} = 2x.\end{aligned}

each layer alone is easy

\begin{aligned}&\text{3. Multiply: } \frac{dy}{dx} = \frac{dy}{du}\cdot\frac{du}{dx} = \cos u \cdot 2x.\end{aligned}

rates along the chain multiply

\begin{aligned}&\text{4. Substitute } u = x^2 \text{ back: } \frac{dy}{dx} = 2x\cos(x^2). \\ &\quad \text{answer in terms of } x\end{aligned}

With practice, steps 1–4 compress into one glance (“derivative of the outside, times derivative of the inside”), but when an expression nests three layers deep — e^{\sqrt{1+x^2}}, say — returning to the explicit ritual is what keeps the factors straight: outer e^u, middle \sqrt v, inner 1 + x^2, so the derivative is e^{\sqrt{1+x^2}} \cdot \frac{1}{2\sqrt{1+x^2}} \cdot 2x.

What the derivative sees#

The derivative is a shape-reader. Where f' > 0 the graph rises; where f' < 0 it falls; where f' = 0 the graph pauses — a candidate maximum, minimum, or plateau.

The second derivative f'' reads curvature: f'' > 0 means bending upward (a smile, concave up), f'' < 0 bending downward (a frown). At a critical point, f'' > 0 confirms a minimum, f'' < 0 a maximum.

Key idea — optimization. To find the best of anything: express it as a function, differentiate, set to zero, check the candidates (and the endpoints of the domain). This five-step ritual solves an astonishing share of applied mathematics, from maximizing enclosed area to minimizing production cost.

Worked example. A farmer has 100 m of fence for a rectangular pen against a wall (no fence needed on the wall side). Area A = x(100 - 2x) where x is the width. A' = 100 - 4x = 0 gives x = 25, so the pen is 25 \times 50 m and A'' = -4 < 0 confirms a maximum: 1250 m².

A limit-cracking bonus: L’Hôpital’s rule#

Derivatives repay their debt to limits. When a limit lands on the indeterminate form \frac{0}{0} (or \frac{\infty}{\infty}), L’Hôpital’s rule says: differentiate top and bottom separately and try again —

\begin{aligned}\lim_{x\to a}\frac{f(x)}{g(x)} = \lim_{x\to a}\frac{f'(x)}{g'(x)} \\ \text{(when the right side exists, and } \tfrac{f}{g} \to \tfrac00 \text{ or } \tfrac{\infty}{\infty}\text{)}.\end{aligned}

Worked slowly, with the checks that make it legal:

Evaluate \lim_{x\to0}\frac{1 - \cos x}{x^2}.

\begin{aligned}&\text{1. Check the form: top} \to 1 - 1 = 0, \text{ bottom} \to 0. && \tfrac00 \text{ — rule applies}\end{aligned}

2. Differentiate top and bottom separately: \lim_{x\to0}\frac{\sin x}{2x}

3. Still \tfrac00 — check again, apply again: \lim_{x\to0}\frac{\cos x}{2} = \frac{1}{2}.

Two warnings the forgetful reader needs most. First, check the form every time: applied to a limit that is not indeterminate, the rule gives garbage (\lim_{x\to1}\frac{x^2}{x} = 1, but differentiating would say \frac{2x}{1} \to 2 — wrong, because the original was \frac{1}{1}, not \frac00). Second, it is quotient-by-quotient differentiation of numerator and denominator separately — not the quotient rule. And a pleasing footnote: the rule was published in the Marquis de l’Hôpital’s 1696 textbook (the world’s first calculus text) but discovered by Johann Bernoulli, whom the Marquis was paying for mathematics lessons — perhaps history’s most successful tuition fee.

Worked example (related rates). Air inflates a spherical balloon at 100 cm³/s. How fast is the radius growing when r = 5 cm? Volume V = \frac{4}{3}\pi r^3, so \frac{dV}{dt} = 4\pi r^2 \frac{dr}{dt} — the chain rule connecting the two rates. At r=5: 100 = 4\pi(25)\frac{dr}{dt}, so \frac{dr}{dt} = \frac{1}{\pi} \approx 0.32 cm/s. The balloon slows visibly as it grows: same air, spread over more surface.

In the wild. The derivative is the most industrially important idea in this book. Marginal cost and marginal revenue in economics are derivatives; velocity sensors differentiate position; PID controllers steering drones and thermostats act on a derivative term. Above all: training a neural network means computing \frac{\partial L}{\partial \theta} for millions of parameters \theta — Part IV shows that “backpropagation” is nothing but this chapter’s chain rule, applied wholesale.

If you keep one thing from this chapter: The derivative is the limit of secant slopes: instantaneous rate of change. The chain rule multiplies rates along a chain — remember it; it will train neural networks in Chapter 22.