Let X⊆R,f:X→R be a function and x0∈X be a limit point of X. The function f is differentiable at x0 if and only if the limit of the difference quotient x−x0f(x)−f(x0) as x→x0 exists.
Let X⊆R,f:X→R be a function and x0∈X be a limit point of X at which f is differentiable. The derivative of f at x0, written dxdf(x0) or f′(x0), is the limit of the difference quotient.
dxdf(x0):=x→x0limx−x0f(x)−f(x0)
If the limit does not exist then dxdf(x0) is undefined and f is not differentiable at x0.
Let X⊆R and f:X→R be a function. The function f is differentiable on X if and only if f is differentiable at every x0∈X.
Let X⊆R and f:X→R be differentiable on X. The derivative of f is the function f′:X→R that maps each x∈X to the derivative of f at x. Writing the nearby point as x+h with step size h→0 gives the equivalent form
dxdf(x)=h→0limhf(x+h)−f(x).
The derivative is also written dxdf(x).
Let X⊆R,f:X→R be a function and x0∈X. If f is differentiable at x0 then f is continuous at x0, so a function that is differentiable on X is continuous on X. The converse is false: f(x)=∣x∣ is continuous at 0 but not differentiable there, since x∣x∣ is 1 for x>0 and −1 for x<0 and so has no limit as x→0.
Let X⊆R and f:X→R be differentiable on X. The second derivative of f at x, written f′′(x)=dx2d2f(x), is the derivative of f′ at x wherever f′ is differentiable. The k-th derivative of f, written f(k)(x)=dxkdkf(x), is defined recursively as the derivative of f(k−1) with f(0)=f.
Let X⊆R,f:X→R be a function and k∈N. The function f is k times continuously differentiable on X, written f∈Ck(X), if and only if f(k) exists and is continuous on X. The set C0(X) is the set of continuous functions on X. The function f is smooth on X, written f∈C∞(X), if and only if f(k) exists for every k∈N.
Let X⊆R,f,g:X→R be differentiable at x∈X and c∈R. Derivatives of constants, sums and scalar multiples satisfy the following rules.
Let n∈N. The power rule states that the derivative of xn is n times xn−1.
dxdxn=nxn−1
The same formula holds for any real exponent n when x>0. Together with the rules for sums and scalar multiples, every polynomial is differentiable on R.
Let X⊆R and f,g:X→R be differentiable at x∈X. The product rule states that fg is differentiable at x with
dxd(f(x)g(x))=f′(x)g(x)+f(x)g′(x).
Let X⊆R and f,g:X→R be differentiable at x∈X with g(x)=0. The quotient rule states that gf is differentiable at x with
dxd(g(x)f(x))=g(x)2f′(x)g(x)−f(x)g′(x).
Let X,Y⊆R,f:X→Y be differentiable at x∈X and g:Y→R be differentiable at f(x). The chain rule states that the composition g∘f is differentiable at x with
dxdg(f(x))=g′(f(x))f′(x).
Let X be an interval and f:X→R be continuous and strictly monotonic. The inverse function rule states that if f is differentiable at x with f′(x)=0 then the inverse function f−1 is differentiable at y=f(x) with
dydf−1(y)=f′(f−1(y))1.
Let X⊆R,f:X→R be a function and x0∈X. The point x0 is a local maximum of f if and only if there exists δ>0 such that f(x)≤f(x0) for all x∈X with ∣x−x0∣<δ. The point x0 is a local minimum of f if and only if there exists δ>0 such that f(x)≥f(x0) for all x∈X with ∣x−x0∣<δ.
Let X⊆R,f:X→R be a function and x0∈X. The point x0 is a global maximum of f if and only if f(x)≤f(x0) for all x∈X. The point x0 is a global minimum of f if and only if f(x)≥f(x0) for all x∈X.
Let X⊆R,f:X→R be a function and x0∈X. The point x0 is an extremum of f if and only if it is a local or global maximum or minimum of f.
Let X⊆R and x0∈X. The point x0 is an interior point of X if and only if some open interval (x0−δ,x0+δ) with δ>0 is contained in X.
Let X⊆R,f:X→R be a function and x0∈X be an interior point of X at which f is differentiable. Fermat’s theorem states that if x0 is a local extremum of f then the derivative of f at x0 is zero.
f′(x0)=0
Let X⊆R,f:X→R be a function and x0∈X. The point x0 is a stationary point of f if and only if f is differentiable at x0 with f′(x0)=0. Stationary points have the following properties:
a stationary point need not be an extremum: f(x)=x3 has f′(0)=0 but 0 is neither a local maximum nor a local minimum;
an extremum that is not an interior point, or at which f is not differentiable, need not be a stationary point;
the extrema of f on [a,b] are found among the stationary points, the endpoints and the points of non-differentiability.
Let f:[a,b]→R be continuous on [a,b] and differentiable on (a,b) with f(a)=f(b).Rolle’s theorem states that the derivative of f is zero at some point strictly between a and b.
∃c∈(a,b):f′(c)=0
Let f:[a,b]→R be continuous on [a,b] and differentiable on (a,b). The mean value theorem states that the derivative of f equals the average rate of change of f over [a,b] at some point strictly between a and b.
∃c∈(a,b):f′(c)=b−af(b)−f(a)
Let X be an interval and f:X→R be differentiable. The mean value theorem has the following consequences:
if f′(x)=0 for all x∈X then f is constant on X;
if f′(x)≥0 for all x∈X then f is monotonically increasing, and if f′(x)>0 for all x∈X then f is strictly increasing;
if ∣f′(x)∣≤L for all x∈X then f is Lipschitz continuous on X with Lipschitz constant L.
Let X⊆R,f:X→R be a function and x0 be an interior stationary point of f at which f′′(x0) exists. The second derivative test states that x0 is a local minimum if f′′(x0)>0 and a local maximum if f′′(x0)<0. If f′′(x0)=0 the test is inconclusive.
Let f,g be differentiable on an open interval containing x0, except possibly at x0 itself, with g′(x)=0 on that interval away from x0.L’Hôpital’s rule states that if the limit of the ratio of derivatives exists, and either limx→x0f(x)=limx→x0g(x)=0 or ∣g(x)∣ grows without bound as x→x0, then the limit of the ratio of the functions equals the limit of the ratio of their derivatives.
x→x0limg(x)f(x)=x→x0limg′(x)f′(x)
Here ∣g(x)∣ grows without bound as x→x0 means that for every M>0 there exists δ>0 with ∣g(x)∣>M whenever 0<∣x−x0∣<δ.
Let X be an interval, x0∈X and f:X→R be k times differentiable at x0. The Taylor polynomial of degree k of f at x0 is the polynomial whose derivatives at x0 up to order k agree with those of f.
Tk(x):=i=0∑ki!f(i)(x0)(x−x0)i
Let X be an interval, x0∈X and f:X→R be k times differentiable at x0 with Taylor polynomial Tk. The remainder of the Taylor polynomial is the difference Rk(x):=f(x)−Tk(x).
Let X be an interval, x0∈X and f:X→R be k times differentiable at x0.Taylor’s theorem states that the remainder vanishes faster than (x−x0)k as x→x0,
x→x0lim(x−x0)kRk(x)=0,
and that if f is k+1 times differentiable on the open interval between x0 and x then there exists c strictly between x0 and x with
Rk(x)=(k+1)!f(k+1)(c)(x−x0)k+1.
The case k=0 is the mean value theorem.
Let X be an interval, x0∈X and f:X→R be smooth at x0. The Taylor series of f at x0 is the power series centred at x0 whose coefficients are the derivatives of f at x0 divided by factorials.
i=0∑∞i!f(i)(x0)(x−x0)i
Let X be an interval, x0∈X and f:X→R be smooth at x0. The function f is analytic at x0 if and only if its Taylor series at x0 converges to f(x) for all x in some open interval containing x0. A smooth function need not be analytic, since its Taylor series may converge to a different function.
Let ∑n=0∞an(x−c)n be a power series with radius of convergence R>0. The term-by-term derivative of the power series is the power series obtained by differentiating each term.
n=1∑∞nan(x−c)n−1
A power series is differentiable on (c−R,c+R), its derivative is its term-by-term derivative and the term-by-term derivative has the same radius of convergence.
dxdn=0∑∞an(x−c)n=n=1∑∞nan(x−c)n−1
Repeating this, a power series is smooth on its open interval of convergence, its coefficients are ak=k!f(k)(c), so the series is its own Taylor series at c, and it is analytic there. Differentiating the exponential, cosine and sine series term by term gives
exp′=exp,cos′=−sin,sin′=cos.
By the inverse function rule, the derivative of the natural logarithm is dxdlogex=x1 for x>0. Since exp′=exp>0 on R, the exponential function is strictly increasing, and the real power ax=exp(xlogea) has derivative axlogea for a>0.
Let x=(x1,…,xn)∈Rn and y∈Rn. The Euclidean distance between x and y is the square root of the sum of the squared differences of their coordinates.
∥x−y∥:=i=1∑n(xi−yi)2
The Euclidean length of x, written ∥x∥, is the Euclidean distance from x to 0.
Let X⊆Rn. The set X is open if and only if every point x∈X has a δ>0 such that every y with ∥y−x∥<δ lies in X.
Let X⊆Rn be open, f:X→R be a function, x∈X,i∈{1,…,n} and ei∈Rn be the i-th standard basis vector. The partial derivative of f with respect to xi at x is the derivative of f along the i-th coordinate with the other coordinates held fixed.
∂xi∂f(x):=h→0limhf(x+hei)−f(x)
Let X⊆Rn be open, f:X→R be a function and x∈X be a point at which all partial derivatives of f exist. The gradient of f at x is the vector of partial derivatives.
∇f(x):=(∂x1∂f(x),…,∂xn∂f(x))
Let X⊆Rn be open, f:X→R be a function and x0∈X be a point at which the gradient of f exists. The point x0 is a stationary point of f if and only if ∇f(x0)=0. For n=1 this is the earlier definition of a stationary point.
Let X⊆Rn be open, f:X→R be a function, x∈X and v∈Rn. The directional derivative of f at x in the direction v is the derivative of f along the line through x in the direction v.
Dvf(x):=h→0limhf(x+hv)−f(x)
The partial derivatives are the directional derivatives along the standard basis vectors.
Let X⊆Rn be open, f:X→R be a function and x∈X. The function f is differentiable at x if and only if there exists g∈Rn such that the error of the linear approximation f(x)+∑i=1ngihi vanishes faster than ∥h∥.
h→0lim∥h∥∣f(x+h)−f(x)−∑i=1ngihi∣=0
Differentiability of a function of several variables has the following properties:
if f is differentiable at x then g=∇f(x),f is continuous at x and every directional derivative exists with Dvf(x)=∑i=1n∂xi∂f(x)vi;
the existence of all partial derivatives at x does not imply differentiability or even continuity at x.
Let X⊆Rn be open and f:X→R be a function. The function f is continuously differentiable on X, written f∈C1(X), if and only if all partial derivatives of f exist and are continuous on X. A continuously differentiable function is differentiable on X.
Let X⊆Rn be open and f=(f1,…,fm):X→Rm be a function whose components fj:X→R are differentiable at x∈X. The Jacobian matrix of f at x is the m×n matrix Jf(x) whose entry in row j and column i is the partial derivative of the j-th component with respect to the i-th coordinate.
[Jf(x)]ji:=∂xi∂fj(x)
The rows of the Jacobian matrix are the gradients of the components of f.
Let X⊆Rn and Y⊆Rm be open, f:X→Y be differentiable at x∈X and g=(g1,…,gp):Y→Rp be differentiable at f(x). The multivariate chain rule states that g∘f is differentiable at x with
Entry by entry this is the product of the Jacobian matrices, Jg∘f(x)=Jg(f(x))Jf(x). In particular, if x(t)=(x1(t),…,xn(t)) is differentiable at t and f is differentiable at x(t) then
dtdf(x(t))=i=1∑n∂xi∂f(x(t))dtdxi(t).
Let X⊆Rn be open, f:X→R be a function and i,j∈{1,…,n}. The second-order partial derivative of f with respect to xi and then xj is the partial derivative of ∂xi∂f with respect to xj.
∂xj∂xi∂2f(x):=∂xj∂(∂xi∂f(x))
Let X⊆Rn be open, f:X→R be a function and x∈X be a point at which all second-order partial derivatives of f exist. The Hessian matrix of f at x is the n×n matrix Hf(x) of second-order partial derivatives.
[Hf(x)]ij:=∂xi∂xj∂2f(x)
Let X⊆Rn be open and f:X→R be a function. The function f is twice continuously differentiable on X, written f∈C2(X), if and only if all second-order partial derivatives of f exist and are continuous on X.
Let X⊆Rn be open and f∈C2(X).Schwarz’s theorem states that the order of differentiation does not matter, so the Hessian matrix is symmetric.
∂xi∂xj∂2f(x)=∂xj∂xi∂2f(x)∀i,j
Let X⊆Rn be open, f∈C2(X),x∈X and h∈Rn with x+h∈X. The second-order Taylor expansion of f at x states that
where the remainder satisfies limh→0∥h∥2R2(h)=0. The first-order terms are the directional derivative Dhf(x) and the second-order terms are the quadratic form 21h⊺Hf(x)h of the Hessian matrix.