Module 37: Gradients, directional derivatives, and tangent planes

A. Prerequisites and observable goals#

B. The central idea: the derivative is a linear map of displacements#

For a differentiable scalar function, the first-order change is $\Delta f\approx f_x\Delta x+f_y\Delta y$. The gradient packages the coefficients into a vector, $\grad f=\langle f_x,f_y\rangle$. A direction chooses an input displacement; a dot product extracts the corresponding rate. The gradient belongs to the input space, while the tangent plane belongs to the graph in one higher dimension.

C. Directional rates and geometric normals#

For a unit vector $\vv u$, define

If $f$ is differentiable, then $D_{\vv u}f=\grad f\cdot\vv u$. Unit length makes this a rate per unit distance. With a nonunit vector $\vv v$, the same composition derivative is a rate per unit parameter and scales by $\|\vv v\|$.

At a point with nonzero gradient, the greatest directional rate is $\|\grad f\|$, in direction $\grad f/\|\grad f\|$; the least is its negative. Directions tangent to a smooth level set have dot product zero with the gradient. A zero gradient gives zero first-order rate in every direction but does not determine whether the point is an extremum.

For a differentiable graph $z=f(x,y)$, the tangent plane at $(a,b,f(a,b))$ is

For a regular implicit level surface $F(x,y,z)=c$ with $\grad F(P)\ne0$, a tangent plane is $\grad F(P)\cdot(\vv r-P)=0$. These use different gradient dimensions: two components for a graph’s scalar function, three for an implicit function of three inputs. [O3Grad]

D. Why the gradient gives steepest increase and a level normal#

The Cauchy–Schwarz inequality gives $\grad f\cdot\vv u\le\|\grad f\|\|\vv u\|=\|\grad f\|$ for unit $\vv u$. Equality occurs when the direction aligns with the nonzero gradient. This conclusion uses the Euclidean length convention and compatible coordinate scales; rescaling the input axes changes what counts as a unit direction.

If a differentiable path $\vv r(t)$ stays on a level set, then $f(\vv r(t))=c$. The chain rule gives $\grad f\cdot\vv r'=0$, proving perpendicularity to the tangent. This is a theorem under differentiability and regularity conditions, not a conclusion from a visually perpendicular arrow alone.

E. A point-and-rate workflow#

Identify the input point and evaluate the gradient there. If a direction is specified by two points, subtract them in the requested order and normalize. Take the dot product, keeping units. For a tangent plane, compute the function’s height as well as the partial slopes; do not confuse that height with the plane’s intercept. For an approximation, insert small increments into the linear model and mark the result approximate. A large displacement may leave the local model inaccurate.

F. Worked examples#

A directional derivative. For $f=x^2+2y^2$ at $(1,1)$, $\grad f=\langle2,4\rangle$. In the direction $\langle3,4\rangle$, use $\vv u=\langle3/5,4/5\rangle$, giving $D_{\vv u}f=22/5$. Without normalization the dot product would be $22$, a rate for a parameter step five times as long.

The tangent-line lesson extends to a plane. At $(1,1)$, the height is $3$, so the graph’s tangent plane is $z=3+2(x-1)+4(y-1)=2x+4y-3$. The height $3$ is not the vertical-axis intercept, just as the earlier point height $-1$ did not make the tangent line $y=3x-1$.

A height-map application. If a game terrain is $z=f(x,y)$, a normal to its tangent plane is $\langle-f_x,-f_y,1\rangle$. Normalizing it gives a unit normal for geometric calculations. This follows from the plane equation; it is not the two-component gradient padded with an arbitrary third value.

Computing bridge: A gradient is a direction of change, not an update policy#

For $f(x,y)=x^2+4y^2$, $\nabla f=(2x,8y)$ tells the local directional rate through a dot product. Choosing $w_{k+1}=w_k-\alpha\nabla f(w_k)$ adds an algorithm and a step size that are not contained in the gradient’s definition. A step can overshoot or behave badly on a poorly scaled problem. A06 derives a simple stability condition for gradient steps, and T03 distinguishes differentiating an objective from successfully training a model.

G. Independent practice#

F foundation C standard course work T transfer S optional stretch L runnable lab or project. Attempt the complete set before opening its solutions.

H. Complete solutions: Module 37#

Compare the first step where your work differs. A corrected answer is not yet an independent success.

Open every solution

I. Retention and readiness#

Retrieve M08.02 and M36.08. Explain why the gradient formula for directional derivatives requires differentiability, and why a tangent plane requires a point height as well as partial slopes. Advance when you can normalize a direction and audit its orientation independently.