Calculus 3D (Derivatives & Gradients)

Question 1

What does the gradient vector represent?

Accepted Answer

The direction of steepest ascent — The gradient vector points in the direction of the greatest rate of increase (steepest ascent) of the function.

Question 2

Which rule is used to compute derivatives of composite functions?

Accepted Answer

Chain Rule — The Chain Rule is essential for computing derivatives of composite functions, which is the mathematical basis for backpropagation in neural networks.

Question 3

The derivative of a function at a point represents…

Accepted Answer

Its instantaneous rate of change — The derivative measures how fast the output changes per unit change in input — the slope used to know which way to adjust parameters.

Question 4

The GRADIENT of a multivariable function is…

Accepted Answer

A vector of partials, steepest ascent — ∇f collects ∂f/∂xᵢ for each variable; it points uphill. Gradient descent therefore steps in the NEGATIVE gradient direction.

Question 5

In gradient descent, parameters are updated by…

Accepted Answer

Subtracting a learning-rate-scaled gradient — Moving opposite the gradient reduces the loss; the learning rate η controls step size.

Question 6

The CHAIN RULE is essential to deep learning because it…

Accepted Answer

Propagates gradients through composed, nested functions — Backprop applies the chain rule layer by layer to propagate the loss gradient back to every weight: dL/dw = product of local derivatives.

Question 7

A partial derivative ∂f/∂x measures the change in f when…

Accepted Answer

Only x changes; all other variables stay fixed — It isolates the effect of one variable. The gradient assembles all partials into a vector.

Question 8

The learning rate η controls…

Accepted Answer

The magnitude of each step taken during gradient descent — Too large η overshoots/diverges; too small η converges slowly. It's one of the most important hyperparameters to tune.

Question 9

At a local minimum of a smooth loss, the gradient is…

Accepted Answer

Zero — A flat tangent (∇L = 0) characterizes minima, maxima, and saddle points; the second derivative/Hessian distinguishes them.

Question 10

A CONVEX loss function guarantees that…

Accepted Answer

Any local minimum is the global minimum — Convexity (bowl shape) means gradient descent can't get stuck in a suboptimal local minimum — true for linear/logistic regression, not deep nets.

Mode

Point Location

Display

One gradient-descent update

Test your knowledge

What does the gradient vector represent?

Interview Questions

References & Further Reading

Love this visualization?