Approximating Roots of Continuous Functions

(This won't be on any HW or Exam)

Theorem (Root Theorem)

Let \( f \) be a continuous function. If \( f(a) < 0 \) and \( f(b) > 0 \) (or vice versa), then \( f \) has a root on the interval \( [a, b] \).

Why?

Sketch of a continuous curve from point A below the x-axis to point B above it, crossing at a root.
Figure 1
A hand-drawn sketch of a coordinate system with a horizontal x-axis and a short vertical y-axis. A continuous S-shaped curve starts at a dot labeled \( A = (a, f(a)) \) located below the x-axis, rises through the x-axis, and ends at a dot labeled \( B = (b, f(b)) \) above the x-axis. Bracket marks on the x-axis label the endpoints \( a \) on the left and \( b \) on the right, indicating the interval \( [a, b] \). A red X marks the point where the curve crosses the x-axis, and a red arrow points to this crossing, identifying it as the location of the root.

Since \( f \) is continuous, we need to get from point \( A \) to point \( B \) without lifting our pen (i.e., without breaking continuity). So \( f \) needs to cross the \( x \)-axis eventually, and that will be the location of the root.

Corollary (Intermediate Value Theorem)

Let \( f \) be a continuous function. If \( f(a) = c \) and \( f(b) = d \), then the interval \( [\min(c, d), \max(c, d)] \) is in the range of \( f \). That is, if \( y \) is between \( c \) and \( d \), there is an \( x \in [a, b] \) where \( f(x) = y \).

Why? Apply the root theorem to \( g(x) = f(x) - y \).

Example 1

Show that \( f(x) = x^2 - 2 \) has a root in the interval \( [0, 2] \).

Solution:

Graph of the parabola crossing the x-axis between 0 and 2.
Figure 2
A small hand-drawn graph of the parabola \( f(x) = x^2 - 2 \). The vertical axis is marked with the value 2 near the top and \( -2 \) below the origin, showing the y-intercept at \( -2 \). The horizontal axis is marked with 2 to the right of the origin, and an open circle sits above \( x = 2 \) at height 2. The curve dips to its minimum at \( -2 \) and rises, crossing the x-axis at a point labeled with an upward arrow to \( \sqrt{2} \), which lies between 0 and 2.

We have \( f(0) = -2 < 0 \) and \( f(2) = 2^2 - 2 = 2 > 0 \). Since \( f \) is continuous, there is a root on the interval \( [0, 2] \).

Question: What's an approximate value of the root?

Answer: Since the root is in \( [0, 2] \), taking the midpoint of the interval (average of the endpoints)

\[ \frac{0 + 2}{2} = 1 \]

is a good guess.

Bisection Method

We can get a better approximation by shrinking our interval.

Successive bisection stages, the resulting interval, and the corresponding approximation of the root
Stage Sketch Interval Approximation of root
Stage 0:
Curve crossing the x-axis with the interval from 0 to 2 marked in red.
Figure 3
A small sketch showing a vertical axis and a horizontal x-axis with an upward-curving parabola-like curve crossing the x-axis. A red horizontal bar with square end brackets spans the x-axis from 0 on the left to 2 on the right, marking the stage 0 interval that contains the root.
\( [0, 2] \) \( x \approx \frac{2 + 0}{2} = 1 \)
Stage 1:
Curve crossing the x-axis with the interval from 1 to 2 marked in red.
Figure 4
A small sketch of the same curve crossing the x-axis. A red bar with end brackets now spans only from 1 to 2 on the x-axis, with the labels 1 above the left bracket and 2 below the right bracket, showing the halved stage 1 interval.
\( [1, 2] \) Notice we halved the length \( x \approx \frac{1 + 2}{2} = 1.5 \)
Stage 2:
Curve crossing the x-axis with the interval from 1 to 1.5 marked in red and 2 to the right.
Figure 5
A small sketch of the same rising curve crossing the x-axis. A short red bar with end brackets spans from 1 to 1.5, labelled 1 above the left bracket and 1.5 below the right bracket, with a black tick marked 2 further to the right, showing the stage 2 interval.
\( [1, 1.5] \) Halved the length of the last interval \( x \approx \frac{1 + 1.5}{2} = 1.25 \)
Stage 3:
Curve crossing the x-axis with the interval from 1.25 to 1.5 marked in red and 2 to the right.
Figure 6
A small sketch of the same rising curve crossing the x-axis. A very short red bar with end brackets spans from 1.25 to 1.5, labelled 1.25 above the left bracket and 1.5 below the right bracket, with a black tick marked 2 further to the right, showing the narrow stage 3 interval.
\( [1.25, 1.5] \) \( x \approx \frac{1.25 + 1.5}{2} = 1.375 \)

Repeat to get an approximation to the precision you want. We bisect (cut in half) each interval to get a smaller interval, so this is called the bisection method.

Algorithm (Bisection Method)

Inputs:

  1. A continuous function \( f \)
  2. Endpoints \( a \) and \( b \) which \( f(a) \) and \( f(b) \) have opposite signs
  3. An error \( \varepsilon > 0 \)

Outputs: An approximate root \( x \) where

\[ \left| x - \left[ \text{True value of the root} \right] \right| < \varepsilon \]

START

  1. \( x \leftarrow \frac{a + b}{2} \)
  2. If \( f(x) = 0 \) OR \( \frac{b - a}{2} < \varepsilon \), then # If a solution has been found
    1. RETURN \( x \)
    2. STOP
  3. If \( f(a) \) and \( f(x) \) have the same sign
    1. \( a \leftarrow x \)
  4. Otherwise,
    1. \( b \leftarrow x \)
  5. Repeat Steps (1) – (4)
Note that we stop the algorithm when the when the absolute error is less than the specified tolerance \((\varepsilon)\). But, there are other stopping conditions based on your situation (e.g., based on the relative error to make it accurate to a certain number of significant digits).

Python Implementation

def bisection(fcn, start, end, TOLERANCE=1e-5/2 , MAX_NUM_ITERATIONS=1000): """ Parameters ---------- fcn : Callable Our continuous function (we called this f). start : float Our left end point (we called this a). end : float Our right end point (we called this b). TOLERANCE : float, optional Our tolerance (this was our epsilon. The default is 1e-5/2, which goes until it is accurate to 5 decimal places. MAX_NUM_ITERATIONS : int, optional The max number of iterations (to prevent an infinite loop). The default is 1000. Returns ------- root: float or None Returns a root of fcn up to the specified level of precision. Returns None if the maximum number of iterations was reached or fcn(start) and fcn(end) have the same sign. """ # Make sure a root is present if (fcn(start)>0 and fcn(end)>0) or (fcn(start)<0 and fcn(end)<0): return None # Start bisection N = 0 while N < MAX_NUM_ITERATIONS: # prevents infinite loop x = (end + start)/2 if fcn(x)==0 or (end - start)/2 < TOLERANCE: return x if (fcn(x)>0 and fcn(start)>0) or (fcn(x)<0 and fcn(start)<0): start = x else: end = x N = N + 1 # If the maximum number of iterations was reached return None if __name__ == "__main__": print("Final Output", bisection(lambda x:x**2 - 2, 0, 2))

Output for \( x^2 - 2 \)

Iteration 1: 1.0 Iteration 2: 1.5 Iteration 3: 1.25 Iteration 4: 1.375 Iteration 5: 1.4375 Iteration 6: 1.40625 Iteration 7: 1.421875 Iteration 8: 1.4140625 Iteration 9: 1.41796875 Iteration 10: 1.416015625 Iteration 11: 1.4150390625 Iteration 12: 1.41455078125 Iteration 13: 1.414306640625 Iteration 14: 1.4141845703125 Iteration 15: 1.41424560546875 Iteration 16: 1.414215087890625 Iteration 17: 1.4141998291015625 Iteration 18: 1.4142074584960938 Iteration 19: 1.4142112731933594 Final Output 1.4142112731933594 In [9]: sqrt(2) Out[9]: 1.4142135623730951

Bound for the Error

If you start on the interval \( [a, b] \), then

\[ \left| [\text{Approximate Value of Root}] - [\text{True Value of Root}] \right| < \frac{b - a}{2^{N}} \]

where \( N \) is the number of bisections.

Example 2

How many rounds of bisection do you need to approximate \( \sqrt{c} \) accurate to 5 decimal places?

Solution: We approximate a root of \( x^2 - c \) on \( [0, c] \) (since we know the root is on that interval).

\[ \left| (\text{Approximation}) - \sqrt{c} \right| < \frac{c - 0}{2^{N}} \overset{\text{WANT}}{<} \frac{10^{-5}}{2} \]

The quantity \( \frac{10^{-5}}{2} \) is what will make it accurate to 5 decimal places.

Need to solve for \(N\) in the inequality:

\[ \begin{aligned} \frac{c}{2^{N}} &< \frac{10^{-5}}{2} \\ \frac{c}{10^{-5}} &< 2^{N-1} \\ \log c - \log(10^{-5}) &< (N-1)\log(2) \\ N &> \frac{\log(c) + 5}{\log(2)} + 1 \end{aligned} \]

The number of bisections \( N \) needed is the integer we obtain when we round up the value
\(\displaystyle \frac{\log(c) + 5}{\log(2)} + 1\).

Example 3

When \( c = 2 \), we get \( N = 19 \), which is what we saw in the Python code.

One Big Caveat

This method is slower compared to others, for example Newton's Method, but the main advantage is we only need continuity to apply this. Another issue is, if there are multiple roots on the interval we bisect, Bisection will only find one of them. This can be solved by finding a "root isolating interval" for all the roots. I.e., an interval that contains one and only one root. For polynomials specifically, we can use Descartes' Rules of Signs to isolate roots, but that discussion is a bit technical for a typical Calculus I student. I just want to point out it is possible.