mathematics-physics-wiki/docs/en/mathematics/linear-algebra/systems-of-linear-equations.md

Systems of linear equations

Definition: a linear equation in n unknowns is an equation of the form

a_1 x_1 + a_2 x_2 + \dots + a_n x_n = b,

with a_i, b \in \mathbb{C} the constants and x_i \in \mathbb{C} the variables for i \in \{1, \dots, n\}.

A linear system of m equations in n unknowns is then a m \times n system of the form

\begin{align*} &a_{11} x_1 + a_{12} x_2 + \dots + a_{1n} x_n = b_1, \ &a_{21} x_1 + a_{22} x_2 + \dots + a_{2n} x_n = b_2, \ &\vdots \ &a_{m1} x_1 + a_{m2} x_2 + \dots + a_{mn} x_n = b_m, \end{align*}

with a_{ij}, b_i \in \mathbb{C} for i \in \{1, \dots, n\} and j \in \{1, \dots, m\}.

A system of linear equations may have one solution, no solution or infinitely many solutions. Think of two lines in euclidean space that may intersect at one point (one solution), are parellel (no solution) or are the same line (infinitely many solutions). If the system has at least one solution that it may be referred to as consistent if it has not than it may be referred to as inconsistent.

Definition: two systems of equations involving the same variables are to be equivalent if they have the same solution set.

A system may be transformed into an equivalent system by

1. changing the order of the equations,
2. multiplying an equation by a non-zero number,
3. and adding a multiple of an equation to another equation.

Definition: a linear system is said to be overdetermined if there are more equations than unknows. A linear system is said to be underdetermined if the opposite is true, there are fewer equations than unknowns.

Overdetermined systems are usually inconsistent and a consistent underdetermined system has always infinitely many solutions.

Definition: a n \times n system is said to be in strict triangular form if in the $k$th equation the coefficients of the first k-1 variables are all zero and the coefficient of x_k is nonzero for k \in \{1, \dots, n\} with n \in \mathbb{N}.

For example the system given by

\begin{align*} 3x_1 + 2x_2 + x_3 &= 1, \ x_2 - x_3 &= 2, \ 2x_3 &= 4, \end{align*}

with x_i \in \mathbb{C} for i \in \{1,2,3\} is in strict triangular form. This system can be solved with back substitution by finding x_3 = 2, then x_2 = 4 and x_1 = -3.

A m \times n system of equations may be represented by a augmented matrix of the form

\left( \begin{array}{cccc|c} a_{11} & a_{12} & \cdots & a_{1n} & b_1 \ a_{21} & a_{22} & \cdots & a_{2n} & b_2 \ \vdots & \vdots & & \vdots & \vdots \ a_{m1} & a_{m2} & \cdots & a_{mn} & b_m\end{array} \right)

with a_{ij}, b_i \in \mathbb{C} for i \in \{1, \dots, n\} and j \in \{1, \dots, m\}.

It may be solved using the following elementary row operations

1. interchange two rows,
2. multiply a row by a nonzero real number,
3. and replace a row by its sum with a multiple of another row.

based of the equivalence transformations.

Row echelon form

Definition: a matrix is said to be in row echelon form

• if the first nonzero entry in each nonzero row is 1, the pivots.
• if row k does not consist entirely of zeros, the number of leading zero entries in row k+1 is greater than the number of leading zero entries in row k.
• if there are rows whose entries are all zero, they are below the rows having nonzero entries.

For example the following matrices are in row echelon form:

\begin{pmatrix} 1 & 4 & 2 \ 0 & 1 & 3 \ 0 & 0 & 1\end{pmatrix}, \qquad \begin{pmatrix} 1 & 2 & 3 \ 0 & 0 & 1 \ 0 & 0 & 0\end{pmatrix}, \qquad \begin{pmatrix} 1 & 3 & 1 & 0 \ 0 & 0 & 1 & 3 \ 0 & 0 & 0 & 0\end{pmatrix}.

Definition: the process of using row operations 1, 2 and 3 to transform a linear system into one whose augmented matrix is in row echelon form is called Gaußian elimination. Obtaining a reduced matrix. Where the variables corresponding to the pivots of reduced matrix will be referred to as lead variables and the variables corresponding to the columns skipped in the process will be referred to as free variables.

Reduced row echelon form

Definition: a matrix is said to be in reduced row echelon form

• if the matrix is in row echelon form.
• if the first nonzero entry in each row is the only nonzero entry in its column.

For example the following matrices are in reduced row echelon form:

\begin{pmatrix} 1 & 0 & 0 \ 0 & 1 & 0 \ 0 & 0 & 1 \end{pmatrix}, \qquad \begin{pmatrix} 1 & 0 & 0 & 3 \ 0 & 1 & 0 & 2 \ 0 & 0 & 1 & 1 \end{pmatrix}, \qquad \begin{pmatrix} 0 & 1 & 2 & 0 \ 0 & 0 & 0 & 1 \ 0 & 0 & 0 & 0 \end{pmatrix}.

The process of using elementary row operations to transform a matrix into reduced row echelon form is called Gauß-Jordan reduction.

Homogeneous systems

Definition: a system of linear equations is said to be homogeneous if the constants on the righthand side are all zero.

Homogeneous are always consistent. Due to their trivial solution; setting all the variables equal to zero.

Theorem: an m \times n homogeneous system of linear equations has a nontrivial solution if n > m.

??? note "Proof:"

Since a homogeneous system is always consistent, the row echelon form of the matrix can have at most $m$ nonzero rows. Thus there are at most $m$ lead variables. Since there are $n$ variables altogether $n > m$ there must be some free variables. The free variables can be assigned arbitrary values. For each assignment of values to the free variables, there is a solution of the system.