Gradient Ascent

Gradient ascent is a fun way to find local maxima of multivariate functions. The basic idea is very simple - just follow the gradient vector. As a result, you end up hiking uphill in the steepest possible direction at every moment and eventually find a local maximum.

Here's a visualization where you can hover over the contour diagram to view a path generated by gradient ascent:

This is Matlab's well known peaks function, which has three local maxima, three local minima, and tapers off to zero. It's formula is:

A 3D graph looks like so: