# Extrema *Definition*: for $D \subseteq \mathbb{R}^n$ let $f: D \to \mathbb{R}$ be differentiable and $D$ contains no boundary points (open). A point $\mathbf{x^*} \in D$ is called a critical point for $f$ $\iff \nabla f(\mathbf{x^*}) = \mathbf{0}$. *Definition*: $f$ has (strict) global $\begin{matrix} \text{ maximum } \\ \text{ minimum } \end{matrix}$ in $\mathbf{x^*} \in D$ $\iff \forall \mathbf{x} \in D \backslash \{\mathbf{x^*}\} \Big[f(\mathbf{x^*}) \begin{matrix} (>) \\ \geq \\ \leq \\ (<) \end{matrix} f(\mathbf{x}) \Big]$. *Definition*: $f$ has (strict) local $\begin{matrix} \text{ maximum } \\ \text{ minimum } \end{matrix}$ in $\mathbf{x^*} \in D$ $\iff \exists r_{>0} \forall \mathbf{x} \in D \backslash \{\mathbf{x^*}\} \Big[f(\mathbf{x^*}) \begin{matrix} (>) \\ \geq \\ \leq \\ (<) \end{matrix} f(\mathbf{x}) \;\land\; (0) < \|\mathbf{x} - \mathbf{x^*}\| < r \Big]$ *Theorem*: if $f$ has local $\begin{matrix} \text{ maximum } \\ \text{ minimum } \end{matrix}$ at $\mathbf{x^*} \in D$ then $\mathbf{x^*}$ is a critical point of for $f$. ??? note "*Proof*:" Will be added later. ## A second derivative test *Definition*: suppose $f: \mathbb{R}^n \to \mathbb{R}$ is differentiable with $\mathbf{x} \in \mathbb{R}^n$. The Hessian matrix of $f$ is defined as $$ H_f(\mathbf{x}) := \begin{pmatrix} \partial_{11} f(\mathbf{x}) & \dots & \partial_{1n} f(\mathbf{x}) \\ \vdots & \ddots & \vdots \\ \partial_{n1} f(\mathbf{x}) & \dots & \partial_{nn} f(\mathbf{x}) \end{pmatrix}. $$ *Theorem*: * If $H_f(\mathbf{x^*})$ is positive definite (all eigenvalues are positive), then $f$ has a local minimum at $\mathbf{x^*}$. * If $H_f(\mathbf{x^*})$ is negative definite (all eigenvalues are negative), then $f$ has a local maximum at $\mathbf{x^*}$. * If $H_f(\mathbf{x^*})$ is indefinite (both positive and negative eigenvalues), then $f$ has a saddle point at $\mathbf{x^*}$. * If $H_f(\mathbf{x^*})$ is neither positive nor negative definite, nor indefinite, (eigenvalues equal to zero) this test gives no information. ??? note "*Proof*:" Will be added later. ## Extrema on restricted domains *Theorem*: let $D \subseteq \mathbb{R}^n$ be bounded and closed ($D$ contains all boundary points). Let $f: D \to \mathbb{R}$ be continuous, then $f$ has a global maximum and minimum. ??? note "*Proof*:" Will be added later. **Procedure to find the global maximum and minimum**: * Find critical points in the interior. * Find global extrema on the boundary. * Find the largest/smallest among them. ### Lagrange multipliers *Theorem*: let $f: M \to \mathbb{R}$ and $g: \mathbb{R}^n \to \mathbb{R}$ with $M$ the boundary of $D$ given by $$ M := \big\{\mathbf{x} \in \mathbb{R}^n \;\big|\; g(\mathbf{x}) = 0 \big\} \subseteq D, $$ suppose that there is global maximum or minimum $\mathbf{x^*} \in M$ of $f$ that is not an endpoint of $M$ and $\nabla g(\mathbf{x^*}) \neq \mathbf{0}$. Then there exists a $\lambda^* \in \mathbb{R}$ such that $(\mathbf{x^*}, \lambda^*)$ is a critical point of the Lagrange function $$ L(\mathbf{x}, \lambda) := f(\mathbf{x}) - \lambda g(\mathbf{x}). $$ ??? note "*Proof*:" Will be added later. ### The general case *Theorem*: Let $f: S \to \mathbb{R}$ and $\mathbf{g}: \mathbb{R}^m \to \mathbb{R}^n$ with $m \leq n -1$ restrictions given by $$ S := \big\{\mathbf{x} \in \mathbb{R}^n \;\big|\; \mathbf{g}(\mathbf{x}) = 0 \big\} \subseteq D, $$ suppose that there is global maximum or minimum $\mathbf{x^*} \in S$ of $f$ that is not an endpoint of $S$ and $D \mathbf{g}(\mathbf{x^*}) \neq \mathbf{0}$. Then there exists a $\mathbf{\lambda^*} \in \mathbb{R^m}$ such that $(\mathbf{x^*}, \mathbf{\lambda^*})$ is a critical point of the Lagrange function $$ L(\mathbf{x}, \mathbf{\lambda}) := f(\mathbf{x}) - \big\langle \mathbf{\lambda},\; \mathbf{g}(\mathbf{x}) \big\rangle. $$ ??? note "*Proof*:" Will be added later. #### Example Let $f: M_1 \cap M_2 \to \mathbb{R}$ and $g_{1,2}: \mathbb{R}^n \to \mathbb{R}$ with the restrictions given by $$ M_{1,2} := \big\{\mathbf{x} \in \mathbb{R}^n \;\big|\; g_{1,2}(\mathbf{x}) = 0 \big\} \subseteq D, $$ suppose that there is global maximum or minimum $\mathbf{x^*} \in M_1 \cap M_2$ of $f$ that is not an endpoint of $M_1 \cap M_2$ and $\nabla g_{1,2}(\mathbf{x^*}) \neq \mathbf{0}$. Then there exists a $\lambda_{1,2}^* \in \mathbb{R}$ such that $(\mathbf{x^*}, \lambda_{1,2}^*)$ is a critical point of the Lagrange function $$ L(\mathbf{x}, \lambda_1, \lambda_2) := f(\mathbf{x}) - \lambda_1 g_1(\mathbf{x}) - \lambda_2 g_2(\mathbf{x}). $$