Four small instruments for the ideas in AI01 that move. Change a slider, predict what the picture will do before you let go, then check. Every number is computed live in your browser with the same 64-bit floats Python uses. Back to the book's companion page
Gradient descent on L(w) = (w − 3)², starting at w = 0. Each step multiplies the distance to 3 by 1 − 2η. Try 0.1, 0.5, 0.9, 1.0, then 1.05.
L = ½(w₁² + κ·w₂²). The steep direction caps η at 2/κ; the gentle one then crawls. Raise κ, then find the largest η that still converges. Click the map to move the start.
Estimate the slope of tanh at x = 0.5 by nudging x. Slide h from large to tiny and watch the error fall, bottom out, then climb as rounding takes over.
A gradient arriving at a tanh node leaves multiplied by the slope 1 − tanh²(z). Push z past about 3 and almost nothing gets through.