# Numerical solutions with Newton's method

## An optimization problem

Suppose we want to find the maximum value of $f(x)=x\sin(x)$ over $[0,\pi]$. A glance at graph shows there's exactly one.

So, how would we find that? I guess we just have to solve $$f'(x) = \sin(x) + x\cos(x) = 0.$$ Unfortunately, that's hard!

## Resources for the numerical solutions of equations

There are plenty of tools for solving equations numerically. One of the simplest to use is WolframAlpha. To solve $\sin(x)+x\cos(x)=0$, for example, just type it in!

If you move on in a technical discipline, though, you'll eventually want to use a more programmatic tool. One broadly applicable tool for mathematical exploration and programming is SageMath.

## Newton's method

Newton's method is a technique to find numerical approximations to roots of functions; it is theoretical foundation on which numerical tools like Sage's find_root works. Given an initial guesss $x_1$, Newton's method improves this guess by applying the function $N(x) = x - \frac{f(x)}{f'(x)}.$ This produces $x_2 = N(x_1)$. We then plug that back in to get $x_3$ and continue. More generally, we produce a sequence $(x_k)$ via $x_k=N(x_{k-1})$.

## Example

Let $f(x) = x^3-x-1$. It's evident from a graph that there's one root.

Newton's method works by riding the tangent line from an initial guess. If we note that $x_1=2$ is pretty close to the root, we compute $x_2 = N(x_2)$, where $N(x) = x - \frac{f(x)}{f'(x)} = x - \frac{x^3-x-1}{3x^2-1}.$ Thus, $x_2 = N(2) = 2-5/11 \approx 1.54545$. Geometrically, this point is obtained by riding the tangent line to the $x$-axis:

If we do that again, we end up even closer to the root:

That's why we iterate!

## Performing Newton's method

Here's how we can apply Newton's method to the previous example using Sage.

## Exercises

1. Use a numerical tool to solve the following equations. Be sure to find all solutions
1. $x^5-x-1 = 0$
2. $x^5-2x-1 = 0$
3. $\sin(3x) = x/2$
2. For each of the following functons, take three Newton steps from the given initial point
1. $f(x) = x^2 - 2$ from $x_1 = 2.0$
2. $f(x) = \sin(x)$ from $x_1 = 3.0$
3. $f(x) = x^5-x-1$ from $x_1 = 1.0$
3. The equation $\sin(x)=x/9$ has 7 solutions. We want to find an approximation to the largest solution. Use a graph to find a good initial approximation for your $x_1$ and apply Newton's method from that point obtain the approximation.