The concept of the derivative is the building block of many topics of calculus. It is important for understanding integrals, gradients, Hessians, and much more.
In this tutorial, you will discover the definition of a derivative, its notation and how you can compute the derivative based upon this definition. You will also discover why the derivative of a function is a function itself.
After completing this tutorial, you will know:
The definition of the derivative of a function
How to compute the derivative of a function based upon the definition
Why some functions do not have a derivative at a point
Let’s get started.
Tutorial Overview
This tutorial is divided into three parts; they are:
The definition and notation used for derivatives of functions
How to compute the derivative of a function using the definition
Why some functions do not have a derivative at a point
What is the Derivative of a Function
In very simple words, the derivative of a function f(x) represents its rate of change and is denoted by either f'(x) or df/dx. Let’s first look at its definition and a pictorial illustration of the derivative.
In the figure, Δx represents a change in the value of x. We keep making the interval between x and (x+Δx) smaller and smaller until it is infinitesimal. Hence, we have the limit (Δ𝑥→0). The numerator f(x+Δx)-f(x) represents the corresponding change in the value of the function f over the interval Δx. This makes the derivative of a function f at a point x, the rate of change of f at that point.
An important point to note is that Δx, the change in x can be negative or positive. Hence:
0<|Δx|< 𝜖,
where 𝜖 is an infinitesimally small value.
About the Notation
The derivative of a function can be denoted by both f'(x) and df/dx. The mathematical giant Newton used f'(x) to denote the derivative of a function. Leibniz, another mathematical hero, used df/dx. So df/dx is a single term, not to be confused with a fraction. It is read as the derivative of a function f with respect to x, and also indicates that x is the independent variable.
Connection with Velocity
One of the most commonly cited examples of derivatives is that of velocity. Velocity is the rate of change of distance w.r.t. time. Hence if f(t) represents the distance travelled at time t, then f'(t) is the velocity at time t. The following sections show various examples of computing the derivative.
Differentiation Examples
The method of finding the derivative of a function is called differentiation. In this section, we’ll see how the definition of the derivative can be used to find the derivative of different functions. Later on, once you are more comfortable with the definition, you can use the defined rules to differentiate a function.
Example 1: m(x) = 2x+5
Let’s start with a simple example of a linear function m(x) = 2x+5. We can see that m(x) changes at a constant rate. We can differentiate this function as follows.
The above figure shows how the function m(x) is changing and it also shows that no matter which value of x, we choose the rate of change of m(x) always remains a 2.
Example 2: g(x) = x^2
Suppose we have the function g(x) given by: g(x) = x^2. The figure below shows how the derivative of g(x) w.r.t. x is calculated. There is also a plot of the function and its derivative in the figure.
As g'(x) = 2x, hence g'(0) = 0, g'(1) = 2, g'(2) = 4 and g'(-1) = -2, g'(-2) = -4
From the figure, we can see that the value of g(x) is very large for large negative values of x. When x < 0, increasing x decreases g(x) and hence g'(x) < 0 for x<0. The graph flattens out for x=0, where the derivative or rate of change of g(x) becomes zero. When x>0, g(x) increases quadratically with the increase in x, and hence, the derivative is also positive.
Example 3: h(x) = 1/x
Suppose we have the function h(x) = 1/x. Shown below is the differentiation of h(x) w.r.t. x (for x ≠0) and the figure illustrating the derivative. The blue curve denotes h(x) and the red curve its corresponding derivative.
Differentiability and Continuity
For example 3, the function h(x) = 1/x is undefined at the point x=0. Hence, its derivative (-1/x^2) is also not defined at x=0. If a function is not continuous at a point, then it does not have a derivative at that point. Below are a few scenarios, where a function is not differentiable:
If the function is not defined at a point
Function does not have a limit at that point
If the function is not continuous at a point
The function has a sudden jump at a point
Following are a few examples:
Extensions
This section lists some ideas for extending the tutorial that you may wish to explore.
Velocity and instantaneous rates of change
Rules for derivatives
Integration
If you explore any of these extensions, I’d love to know. Post your findings in the comments below.
Further Reading
This section provides more resources on the topic if you are looking to go deeper.
Tutorials
Limits and continuity
Evaluating limits
Resources
Additional resources on Calculus Books for Machine Learning
Books
Thomas’ Calculus, 14th edition, 2017. (based on the original works of George B. Thomas, revised by Joel Hass, Christopher Heil, Maurice Weir)
Calculus, 3rd Edition, 2017. (Gilbert Strang)
Calculus, 8th edition, 2015. (James Stewart)
Summary
In this tutorial, you discovered the function derivatives and the fundamentals of function differentiation.
Specifically, you learned:
The definition and notation of a function derivative
How to differentiate a function using the definition
When a function is not differentiable
Do you have any questions? Ask your questions in the comments below and I will do my best to answer.
The post A Gentle Introduction to Function Derivatives appeared first on Machine Learning Mastery.
Read MoreMachine Learning Mastery
