[Chapter 2 Summary] of Linear Algebra

Chapter 2 Matrix Algebra

1. Matrix Operations

Diagonal entries

a. Multiplication

• Definition: $AB = A[b_1 \ b_2 \ \dots \ b_p] = [Ab_1 \ \dots \ Ab_p]$
• Interpretation: $A$ column is a linear combination of the columns of $A$ using $b_i$ weights.
• Row-Column Rule: $(AB)_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \dots + a_{in}b_{nj}$
• Additional Property: $\text{ROW}_i(AB) = \text{ROW}_i(A) \cdot B = a_i B$
• Algebraic Properties:
  • $A(BC) = (AB)C$
  • $A(B+C) = AB + AC$
  • $r(AB) = (rA)B = A(rB)$
• Commutativity: If $AB = BA$, then $A$ and $B$ are said to commute.

b. Power

• $A^k = \underbrace{A \dots A}_{k}$

c. Transpose

• Definition: $A \to A^T$ (Columns formed the corresponding rows)
• Properties:
  • $(A^T)^T = A$
  • $(A+B)^T = A^T + B^T$
  • $(rA)^T = rA^T$
  • $(AB)^T = B^T A^T$ (Important)

d. Inverse

• Definition: If $CA = I$ and $AC = I$, then $C$ is the inverse of $A$ (unique).
• For a $2 \times 2$ Matrix:
  For $\begin{bmatrix} a & b \\ c & d \end{bmatrix}$, if $\det A = ad - bc \neq 0$, then $A$ is invertible:

  $$A^{-1} = \frac{1}{ad-bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$$

• Properties:
  • $(A^{-1})^{-1} = A$
  • $(AB)^{-1} = B^{-1} A^{-1}$
  • $(A^T)^{-1} = (A^{-1})^T$ (Important)
• Way to find $A^{-1}$:
  • Elementary Matrices: Row operation on an identity Matrix.
  • A sequence of $E$: $A \to I_n$ then $I_n \to A^{-1}$. So: $[A \ I] \iff [I \ A^{-1}]$.
• Terminology:
  • Invertible = Nonsingular
  • Noninvertible = Singular
• Invertible Linear Transformation:
  • $S(T(x)) = x$
  • $T(S(x)) = x$

2. Partitioned Matrix (Block Matrix)

• Multiplication: Conformable (The column partition of $A$ is same as the row partition of $B$).
• Column-Row Expansion of $AB$:

  $$AB = [\text{col}_1(A) \ \text{col}_2(A) \ \dots \ \text{col}_n(A)] \begin{bmatrix} \text{row}_1(B) \\ \text{row}_2(B) \\ \vdots \\ \text{row}_n(B) \end{bmatrix} = \text{col}_1(A)\text{row}_1(B) + \dots + \text{col}_n(A)\text{row}_n(B)$$

3. Matrix Factorization

Expresses $A$ as a product of 2 matrices.

The LU Factorization

• $A = LU$
  • $L$: Lower triangular matrix
  • $U$: Upper triangular matrix
  • (Where $A \to$ echelon form without row interchange)
• Useful for solving $Ax = b$:

  $$Ax = b \Rightarrow L(Ux) = b \Rightarrow \begin{cases} Ly = b \\ Ux = y \end{cases}$$

• Algorithm:
  • $U$ is the echelon form: $E_p \dots E_1 A = U \Rightarrow L = (E_p \dots E_1)^{-1}$
  • Note that row operations to reduce $A \to U$ also take $L \to I$.

4. Subspace

A subspace $H$ must satisfy:

1. $0 \in H$
2. If $u, v \in H$, then $u + v \in H$
3. If $u \in H$, then $cu \in H$

• Column Space: $\text{Col } A = \text{span}\{a_1, \dots, a_n\}$ (Linear combinations of all the columns in $A$)
• Null Space: All solutions of the homogeneous equation $Ax = 0$
• Basis: A set (linearly independent) that spans $H$
• Standard Basis: $e_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, e_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, e_3 = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$
• Important Note: The pivot columns of matrix $A$ form a basis of $\text{Col } A$ (DO NOT use echelon form).

5. Coordinate System

• If $x = c_1b_1 + c_2b_2 + \dots + c_pb_p$, then $[x]_B = \begin{bmatrix} c_1 \\ \vdots \\ c_p \end{bmatrix}$ (Coordinate vector of $x$ relative to $B$).
• Dimension: $\dim H$: Number of vectors in any basis of $H$.
• Rank: $\text{rank } A = \dim \text{Col } A$.
• Property (Rank-Nullity Theorem): $\text{rank } A + \dim \text{Nul } A = n$.
• The Basis Theorem: Let $p = \dim H$.
  • A linearly independent set of $p$ vectors in $H$ is automatically a basis for $H$.
  • A set of $p$ vectors that spans $H$ is automatically a basis for $H$.

6. The Invertible Theorem

Let $A$ be an $n \times n$ matrix. The following statements are equivalent ($A$ is Invertible):

Row Operations:

• $A$ is row equivalent to $I_n$
• $A$ has $n$ pivot positions

Equations:

• $Ax = 0$ has only the trivial solution
• $Ax = b$ has at least one solution for each $b$ in $\mathbb{R}^n$
• $\dim \text{Nul } A = 0$

Columns:

• The columns of $A$ form a linearly independent set
• The columns of $A$ form a basis of $\mathbb{R}^n$
• The columns of $A$ span $\mathbb{R}^n$
• $\text{Col } A = \mathbb{R}^n$
• $\text{rank } A = n$

Invertibility / Transpose:

• There exists an $n \times n$ matrix $C$ such that $CA = I$
• There exists an $n \times n$ matrix $D$ such that $AD = I$
• $A^T$ is invertible

Linear Transformation ($T: x \mapsto Ax$):

• $T$ is onto (surjective)
• $T$ is one-to-one (injective)