

# Numerical differentiation: the basics



To compute the derivative of a function numerically, we can use the forward difference formula:

$$\frac{df(x)}{dx} \approx \frac{f(x + \Delta x) - f(x)}{\Delta x}$$

This formula calculates the difference between the function values at $x + \Delta x$ and $x$, and divides it by $\Delta x$ to obtain an approximation of the derivative. The accuracy of this approximation depends on the choice of $\Delta x$.

Let's consider an example. Suppose we want to approximate the derivative of the function $f(x) = x^2$ at the point $x = 2$. We can use the forward difference formula with a small value of $\Delta x$, say $\Delta x = 0.01$.

Consider the function $f(x) = x^2$. We want to approximate the derivative of $f$ at $x = 2$ using the forward difference formula with $\Delta x = 0.01$.

First, we compute the function values at $x = 2 + \Delta x$ and $x = 2$:

$$f(2 + 0.01) = (2 + 0.01)^2 = 4.0101$$

$$f(2) = 2^2 = 4$$

Next, we compute the difference between the function values:

$$\Delta f = f(2 + 0.01) - f(2) = 4.0101 - 4 = 0.0101$$

Finally, we divide the difference by $\Delta x$ to obtain the approximation of the derivative:

$$\frac{df(2)}{dx} \approx \frac{0.0101}{0.01} = 1.01$$

So, our approximation of the derivative of $f$ at $x = 2$ is $1.01$.

We can also use other formulas for numerical differentiation, such as the central difference formula:

$$\frac{df(x)}{dx} \approx \frac{f(x + \Delta x) - f(x - \Delta x)}{2\Delta x}$$

This formula is more accurate than the forward difference formula, as it takes into account the function values at both sides of the point of interest.

Let's consider another example. Suppose we want to approximate the derivative of the function $f(x) = x^3$ at the point $x = 2$ using the central difference formula with $\Delta x = 0.01$.

First, we compute the function values at $x = 2 + \Delta x$ and $x = 2 - \Delta x$:

$$f(2 + 0.01) = (2 + 0.01)^3 = 8.0304$$

$$f(2 - 0.01) = (2 - 0.01)^3 = 7.9696$$

Next, we compute the difference between the function values:

$$\Delta f = f(2 + 0.01) - f(2 - 0.01) = 8.0304 - 7.9696 = 0.0608$$

Finally, we divide the difference by $2\Delta x$ to obtain the approximation of the derivative:

$$\frac{df(2)}{dx} \approx \frac{0.0608}{0.02} = 3.04$$

So, our approximation of the derivative of $f$ at $x = 2$ is $3.04$.

## Exercise
Consider the function $f(x) = e^x$. Use the forward difference formula with $\Delta x = 0.1$ to approximate the derivative of $f$ at $x = 1$.



# Numerical integration: the basics



To compute the definite integral of a function numerically, we can use the trapezoidal rule. The trapezoidal rule approximates the area under the curve by dividing the interval into subintervals and approximating each subinterval as a trapezoid.

Let's consider an example. Suppose we want to approximate the definite integral of the function $f(x) = x^2$ from $x = 1$ to $x = 3$. We can use the trapezoidal rule with $n$ subintervals, where $n$ is a positive integer.

Consider the function $f(x) = x^2$. We want to approximate the definite integral of $f$ from $x = 1$ to $x = 3$ using the trapezoidal rule with $n = 4$ subintervals.

First, we divide the interval $[1, 3]$ into $n = 4$ subintervals:

$$\Delta x = \frac{3 - 1}{4} = 0.5$$

Next, we compute the function values at the endpoints of each subinterval:

$$f(1) = 1^2 = 1$$

$$f(1 + \Delta x) = (1 + 0.5)^2 = 1.25$$

$$f(1 + 2\Delta x) = (1 + 2\cdot 0.5)^2 = 2.25$$

$$f(1 + 3\Delta x) = (1 + 3\cdot 0.5)^2 = 3.25$$

$$f(3) = 3^2 = 9$$

Finally, we compute the trapezoidal rule approximation:

$$\int_1^3 f(x)dx \approx \frac{\Delta x}{2} \left( f(1) + 2f(1 + \Delta x) + 2f(1 + 2\Delta x) + 2f(1 + 3\Delta x) + f(3) \right)$$

$$\int_1^3 f(x)dx \approx \frac{0.5}{2} \left( 1 + 2\cdot 1.25 + 2\cdot 2.25 + 2\cdot 3.25 + 9 \right)$$

$$\int_1^3 f(x)dx \approx 2.5 \left( 1 + 2.5 + 5 + 7.5 + 9 \right)$$

$$\int_1^3 f(x)dx \approx 2.5 \cdot 22$$

$$\int_1^3 f(x)dx \approx 49$$

So, our approximation of the definite integral of $f$ from $x = 1$ to $x = 3$ is $49$.

We can also use other formulas for numerical integration, such as Simpson's rule.

Let's consider another example. Suppose we want to approximate the definite integral of the function $f(x) = x^3$ from $x = 1$ to $x = 3$ using Simpson's rule with $n = 4$ subintervals.

First, we divide the interval $[1, 3]$ into $n = 4$ subintervals:

$$\Delta x = \frac{3 - 1}{4} = 0.5$$

Next, we compute the function values at the endpoints of each subinterval:

$$f(1) = 1^3 = 1$$

$$f(1 + \Delta x) = (1 + 0.5)^3 = 1.125$$

$$f(1 + 2\Delta x) = (1 + 2\cdot 0.5)^3 = 1.875$$

$$f(1 + 3\Delta x) = (1 + 3\cdot 0.5)^3 = 3.375$$

$$f(3) = 3^3 = 27$$

Finally, we compute the Simpson's rule approximation:

$$\int_1^3 f(x)dx \approx \frac{\Delta x}{3} \left( f(1) + 4f(1 + \Delta x) + 2f(1 + 2\Delta x) + 4f(1 + 3\Delta x) + f(3) \right)$$

$$\int_1^3 f(x)dx \approx \frac{0.5}{3} \left( 1 + 4\cdot 1.125 + 2\cdot 1.875 + 4\cdot 3.375 + 27 \right)$$

$$\int_1^3 f(x)dx \approx 0.166667 \left( 1 + 4.5 + 3.75 + 13.5 + 27 \right)$$

$$\int_1^3 f(x)dx \approx 0.166667 \cdot 42.75$$

$$\int_1^3 f(x)dx \approx 7.11875$$

So, our approximation of the definite integral of $f$ from $x = 1$ to $x = 3$ is $7.11875$.

## Exercise
Instructions:

Consider the function $f(x) = e^x$. Use the trapezoidal rule with $\Delta x = 0.1$ to approximate the definite integral of $f$ from $x = 0$ to $x = 2$.

### Solution
None



# Riemann sums: the foundation of numerical integration



To compute the definite integral of a function using Riemann sums, we can follow these steps:

1. Divide the interval of integration into a sequence of subintervals.
2. For each subinterval, approximate the area under the curve by a rectangle with a width equal to the width of the subinterval and a height equal to the function value at a point in the subinterval.
3. Sum the areas of all the rectangles to obtain the approximation of the definite integral.

Let's consider an example. Suppose we want to approximate the definite integral of the function $f(x) = x^2$ from $x = 1$ to $x = 3$ using Riemann sums with $n = 4$ subintervals.

First, we divide the interval $[1, 3]$ into $n = 4$ subintervals:

$$\Delta x = \frac{3 - 1}{4} = 0.5$$

Next, we approximate the area under the curve by a sequence of rectangles:

$$\int_1^3 f(x)dx \approx \sum_{i = 1}^{n} f(1 + (i - 1)\Delta x) \cdot \Delta x$$

$$\int_1^3 f(x)dx \approx \sum_{i = 1}^{4} (1 + (i - 1)\cdot 0.5)^2 \cdot 0.5$$

$$\int_1^3 f(x)dx \approx \sum_{i = 1}^{4} (1 + (i - 1)\cdot 0.5)^2 \cdot 0.5$$

$$\int_1^3 f(x)dx \approx 0.5 \left( 1 + 0.5^2 + 0.75^2 + 1.25^2 \right)$$

$$\int_1^3 f(x)dx \approx 0.5 \cdot 3.5$$

$$\int_1^3 f(x)dx \approx 1.75$$

So, our approximation of the definite integral of $f$ from $x = 1$ to $x = 3$ is $1.75$.

## Exercise
Instructions:

Consider the function $f(x) = e^x$. Use Riemann sums with $\Delta x = 0.1$ to approximate the definite integral of $f$ from $x = 0$ to $x = 2$.

### Solution
None



# Simpson's rule: an improvement on Riemann sums



To compute the definite integral of a function using Simpson's rule, we can follow these steps:

1. Divide the interval of integration into a sequence of subintervals.
2. For each subinterval, approximate the area under the curve by a parabolic segment with a width equal to the width of the subinterval and a height equal to the function value at a point in the subinterval.
3. Sum the areas of all the parabolic segments to obtain the approximation of the definite integral.

Let's consider an example. Suppose we want to approximate the definite integral of the function $f(x) = x^3$ from $x = 1$ to $x = 3$ using Simpson's rule with $n = 4$ subintervals.

First, we divide the interval $[1, 3]$ into $n = 4$ subintervals:

$$\Delta x = \frac{3 - 1}{4} = 0.5$$

Next, we approximate the area under the curve by a sequence of parabolic segments:

$$\int_1^3 f(x)dx \approx \frac{\Delta x}{3} \left( f(1) + 4f(1 + \Delta x) + 2f(1 + 2\Delta x) + 4f(1 + 3\Delta x) + f(3) \right)$$

$$\int_1^3 f(x)dx \approx \frac{0.5}{3} \left( 1 + 4\cdot 1.125 + 2\cdot 1.875 + 4\cdot 3.375 + 27 \right)$$

$$\int_1^3 f(x)dx \approx 0.166667 \left( 1 + 4.5 + 3.75 + 13.5 + 27 \right)$$

$$\int_1^3 f(x)dx \approx 0.166667 \cdot 42.75$$

$$\int_1^3 f(x)dx \approx 7.11875$$

So, our approximation of the definite integral of $f$ from $x = 1$ to $x = 3$ is $7.11875$.

## Exercise
Instructions:

Consider the function $f(x) = e^x$. Use Simpson's rule with $\Delta x = 0.1$ to approximate the definite integral of $f$ from $x = 0$ to $x = 2$.

### Solution
None



# Trapezoidal rule: another approach to numerical integration



To compute the definite integral of a function using the trapezoidal rule, we can follow these steps:

1. Divide the interval of integration into a sequence of subintervals.
2. For each subinterval, approximate the area under the curve by a trapezoid with a width equal to the width of the subinterval and a height equal to the function value at the endpoints of the subinterval.
3. Sum the areas of all the trapezoids to obtain the approximation of the definite integral.

Let's consider an example. Suppose we want to approximate the definite integral of the function $f(x) = x^2$ from $x = 1$ to $x = 3$ using the trapezoidal rule with $n = 4$ subintervals.

First, we divide the interval $[1, 3]$ into $n = 4$ subintervals:

$$\Delta x = \frac{3 - 1}{4} = 0.5$$

Next, we approximate the area under the curve by a sequence of trapezoids:

$$\int_1^3 f(x)dx \approx \frac{\Delta x}{2} \left( f(1) + 2f(1 + \Delta x) + 2f(1 + 2\Delta x) + 2f(1 + 3\Delta x) + f(3) \right)$$

$$\int_1^3 f(x)dx \approx \frac{0.5}{2} \left( 1 + 2\cdot 1.25 + 2\cdot 2.25 + 2\cdot 3.25 + 9 \right)$$

$$\int_1^3 f(x)dx \approx 2.5 \left( 1 + 2.5 + 5 + 7.5 + 9 \right)$$

$$\int_1^3 f(x)dx \approx 2.5 \cdot 22$$

$$\int_1^3 f(x)dx \approx 49$$

So, our approximation of the definite integral of $f$ from $x = 1$ to $x = 3$ is $49$.

## Exercise
Instructions:

Consider the function $f(x) = e^x$. Use the trapezoidal rule with $\Delta x = 0.1$ to approximate the definite integral of $f$ from $x = 0$ to $x = 2$.

### Solution
None



# Applying numerical methods to real-world problems



For example, numerical methods can be used to solve problems in physics, such as finding the electric field around a charged particle or calculating the heat transfer in a material. In engineering, numerical methods can be used to solve problems in fluid dynamics, structural analysis, and computer-aided design. In economics, numerical methods can be used to solve problems in finance, such as calculating the present value of a stream of cash flows or estimating the value of an option.

Let's consider an example. Suppose we want to use numerical methods to solve a problem in physics. We want to find the electric field around a charged particle with a charge of $q = 10^6 C$ and a radius of $R = 10^6 m$. We can use the method of images to solve this problem and obtain the electric field at a distance of $r = 20^6 m$ from the charged particle.

## Exercise
Instructions:

Consider the problem of finding the electric field around a charged particle with a charge of $q = 10^6 C$ and a radius of $R = 10^6 m$. Use numerical methods to solve this problem and obtain the electric field at a distance of $r = 20^6 m$ from the charged particle.

### Solution
None



# Comparing different methods for derivatives and integrals



For example, we can compare the forward difference formula, the central difference formula, and Simpson's rule for numerical differentiation. We can also compare the trapezoidal rule, Simpson's rule, and Riemann sums for numerical integration.

Let's consider an example. Suppose we want to compare the accuracy of the forward difference formula, the central difference formula, and Simpson's rule for numerical differentiation. We will use the function $f(x) = x^2$ and compute the approximations of the derivative at $x = 2$ using each method.

## Exercise
Instructions:

Consider the function $f(x) = x^2$. Use the forward difference formula, the central difference formula, and Simpson's rule to compute the approximations of the derivative of $f$ at $x = 2$.

### Solution
None



# Hands-on examples using Python



For example, we can use the NumPy library to compute numerical approximations of derivatives and integrals. We can also use the SciPy library to compute numerical approximations of derivatives and integrals using various numerical methods.

Let's consider an example. Suppose we want to use Python and the NumPy library to compute the numerical approximation of the derivative of the function $f(x) = x^2$ at $x = 2$.

## Exercise
Instructions:

Consider the function $f(x) = x^2$. Use Python and the NumPy library to compute the numerical approximation of the derivative of $f$ at $x = 2$.

### Solution
None



# Pitfalls and limitations of numerical methods



For example, numerical methods can be sensitive to the choice of $\Delta x$ or $\Delta t$ in the forward difference formula, the central difference formula, and Simpson's rule. Numerical methods can also suffer from round-off errors and truncation errors.

Let's consider an example. Suppose we want to compute the numerical approximation of the derivative of the function $f(x) = x^2$ at $x = 2$ using the forward difference formula with a small value of $\Delta x$.

## Exercise
Instructions:

Consider the function $f(x) = x^2$. Use the forward difference formula with a small value of $\Delta x$ to compute the numerical approximation of the derivative of $f$ at $x = 2$.

### Solution
None



# Advanced topics in numerical methods for derivatives and integrals



For example, adaptive numerical methods can automatically adjust the step size in numerical integration to improve the accuracy of the approximation. Multidimensional numerical integration can be used to compute numerical approximations of multiple integrals, such as the volume of a region in space. Numerical methods for solving differential equations can be used to solve problems in physics, engineering, and economics that involve time-dependent variables.

Let's consider an example. Suppose we want to use numerical methods to solve the initial value problem for the first-order ordinary differential equation $y'(x) = f(x, y)$ with initial condition $y(a) = y_0$. We can use the method of Euler to approximate the solution of the differential equation.

## Exercise
Instructions:

Consider the first-order ordinary differential equation $y'(x) = f(x, y)$ with initial condition $y(a) = y_0$. Use the method of Euler to approximate the solution of the differential equation.

### Solution
None

