# Extrema

*Definition*: for $D \subseteq \mathbb{R}^n$ let $f: D \to \mathbb{R}$ be differentiable and $D$ contains no boundary points (open). A point $\mathbf{x^*} \in D$ is called a critical point for $f$ $\iff \nabla f(\mathbf{x^*}) = \mathbf{0}$.

*Definition*: $f$ has (strict) global $\begin{matrix} \text{ maximum } \\ \text{ minimum } \end{matrix}$ in $\mathbf{x^*} \in D$ $\iff \forall \mathbf{x} \in D \backslash \{\mathbf{x^*}\} \Big[f(\mathbf{x^*}) \begin{matrix} (>) \\ \geq \\ \leq \\ (<) \end{matrix} f(\mathbf{x}) \Big]$.

*Definition*: $f$ has (strict) local $\begin{matrix} \text{ maximum } \\ \text{ minimum } \end{matrix}$ in $\mathbf{x^*} \in D$ $\iff \exists r_{>0} \forall \mathbf{x} \in D \backslash \{\mathbf{x^*}\} \Big[f(\mathbf{x^*}) \begin{matrix} (>) \\ \geq \\ \leq \\ (<) \end{matrix} f(\mathbf{x}) \;\land\;  (0) < \|\mathbf{x} - \mathbf{x^*}\| < r \Big]$

*Theorem*: if $f$ has local $\begin{matrix} \text{ maximum } \\ \text{ minimum } \end{matrix}$ at $\mathbf{x^*} \in D$ then $\mathbf{x^*}$ is a critical point of for $f$.

??? note "*Proof*:"

    Will be added later.

## A second derivative test

*Definition*: suppose $f: \mathbb{R}^n \to \mathbb{R}$ is differentiable with $\mathbf{x} \in \mathbb{R}^n$. The Hessian matrix of $f$ is defined as

$$
    H_f(\mathbf{x}) := \begin{pmatrix} \partial_{11} f(\mathbf{x}) & \dots & \partial_{1n} f(\mathbf{x}) \\ \vdots & \ddots & \vdots \\ \partial_{n1} f(\mathbf{x}) & \dots & \partial_{nn} f(\mathbf{x}) \end{pmatrix}.
$$

*Theorem*:

* If $H_f(\mathbf{x^*})$ is positive definite (all eigenvalues are positive), then $f$ has a local minimum at $\mathbf{x^*}$.
* If $H_f(\mathbf{x^*})$ is negative definite (all eigenvalues are negative), then $f$ has a local maximum at $\mathbf{x^*}$.
* If $H_f(\mathbf{x^*})$ is indefinite (both positive and negative eigenvalues), then $f$ has a saddle point at $\mathbf{x^*}$.
* If $H_f(\mathbf{x^*})$ is neither positive nor negative definite, nor indefinite, (eigenvalues equal to zero) this test gives no information.

??? note "*Proof*:"

    Will be added later.

## Extrema on restricted domains

*Theorem*: let $D \subseteq \mathbb{R}^n$ be bounded and closed ($D$ contains all boundary points). Let $f: D \to \mathbb{R}$ be continuous, then $f$ has a global maximum and minimum.

??? note "*Proof*:"

    Will be added later.

**Procedure to find the global maximum and minimum**:

* Find critical points in the interior.
* Find global extrema on the boundary.
* Find the largest/smallest among them.

### Lagrange multipliers

*Theorem*: let $f: M \to \mathbb{R}$ and $g: \mathbb{R}^n \to \mathbb{R}$ with $M$ the boundary of $D$ given by

$$
    M := \big\{\mathbf{x} \in \mathbb{R}^n \;\big|\; g(\mathbf{x}) = 0  \big\} \subseteq D,
$$

suppose that there is global maximum or minimum $\mathbf{x^*} \in M$ of $f$ that is not an endpoint of $M$ and $\nabla g(\mathbf{x^*}) \neq \mathbf{0}$. Then there exists a $\lambda^* \in \mathbb{R}$ such that $(\mathbf{x^*}, \lambda^*)$ is a critical point of the Lagrange function

$$
    L(\mathbf{x}, \lambda) := f(\mathbf{x}) - \lambda g(\mathbf{x}).
$$

??? note "*Proof*:"

    Will be added later.

### The general case

*Theorem*: Let $f: S \to \mathbb{R}$ and $\mathbf{g}: \mathbb{R}^m \to \mathbb{R}^n$ with $m \leq n -1$ restrictions given by

$$
    S := \big\{\mathbf{x} \in \mathbb{R}^n \;\big|\; \mathbf{g}(\mathbf{x}) = 0  \big\} \subseteq D,
$$

suppose that there is global maximum or minimum $\mathbf{x^*} \in S$ of $f$ that is not an endpoint of $S$ and $D \mathbf{g}(\mathbf{x^*}) \neq \mathbf{0}$. Then there exists a $\mathbf{\lambda^*} \in \mathbb{R^m}$ such that $(\mathbf{x^*}, \mathbf{\lambda^*})$ is a critical point of the Lagrange function

$$
    L(\mathbf{x}, \mathbf{\lambda}) := f(\mathbf{x}) - \big\langle \mathbf{\lambda},\; \mathbf{g}(\mathbf{x})  \big\rangle.
$$

??? note "*Proof*:"

    Will be added later.

#### Example

Let $f: M_1 \cap M_2 \to \mathbb{R}$ and $g_{1,2}: \mathbb{R}^n \to \mathbb{R}$ with the restrictions given by

$$
    M_{1,2} := \big\{\mathbf{x} \in \mathbb{R}^n \;\big|\; g_{1,2}(\mathbf{x}) = 0  \big\} \subseteq D,
$$

suppose that there is global maximum or minimum $\mathbf{x^*} \in M_1 \cap M_2$ of $f$ that is not an endpoint of $M_1 \cap M_2$ and $\nabla g_{1,2}(\mathbf{x^*}) \neq \mathbf{0}$. Then there exists a $\lambda_{1,2}^* \in \mathbb{R}$ such that $(\mathbf{x^*}, \lambda_{1,2}^*)$ is a critical point of the Lagrange function

$$
    L(\mathbf{x}, \lambda_1, \lambda_2) := f(\mathbf{x}) - \lambda_1 g_1(\mathbf{x}) - \lambda_2 g_2(\mathbf{x}).
$$