Let F\mathbb{F} be a field and n,p≥1n,p \geq 1 be natural numbers. A matrix A∈F n×pA \in \mathbb{F}^{\,n\times p} over F\mathbb{F} is a collection of npnp scalars aij∈Fa_{ij} \in \mathbb{F} indexed by a row index i=1,…,ni=1,\ldots,n and a column index j=1,…,p.j=1,\ldots,p. The order of AA is n×p,n \times p, where nn is the number of rows and pp is the number of columns. A matrix is written in rectangular form.

A=[a11a12…a1pa21a22…a2p⋮⋮⋱⋮an1an2…anp]A=\begin{bmatrix} a_{11} & a_{12} & \ldots & a_{1p} \\ a_{21} & a_{22} & \ldots & a_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \ldots & a_{np} \end{bmatrix}

Let A∈F n×pA \in \mathbb{F}^{\,n\times p} be a matrix. The matrix AA is square if and only if n=p.n=p. The scalars aiia_{ii} of a square matrix are its diagonal elements.

A column vector a‾∈Fn\underline{a}\in \mathbb{F}^n is a matrix of order n×1n \times 1 and is written in column form.

a‾=[a1a2⋮an]\underline{a} = \begin{bmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \end{bmatrix}

A row vector a‾∈F1×p\underline{a}\in \mathbb{F}^{1\times p} is a matrix of order 1×p1 \times p and is written in row form.

a‾=[a1a2…ap]\underline{a} = \begin{bmatrix} a_1 & a_2 & \ldots & a_p \end{bmatrix}

Let A∈F n×pA \in \mathbb{F}^{\,n\times p} be a matrix. The columns of AA are the pp column vectors a‾1,…,a‾p∈Fn\underline{a}_1,\ldots,\underline{a}_p \in \mathbb{F}^n formed from each column of A,A, and the rows of AA are the nn row vectors formed from each row of A.A.

Let A,B∈F n×pA, B \in \mathbb{F}^{\,n\times p} be matrices of the same order. Matrix addition is the matrix A+B∈F n×pA+B \in \mathbb{F}^{\,n\times p} whose elements are the element-wise sums.

(A+B)ij:=aij+bij(A+B)_{ij} := a_{ij}+b_{ij}

Let A∈F n×pA \in \mathbb{F}^{\,n\times p} be a matrix and λ∈F\lambda \in \mathbb{F} be a scalar. Matrix scalar multiplication is the matrix λA∈F n×p\lambda A \in \mathbb{F}^{\,n\times p} whose elements are the elements of AA multiplied by λ.\lambda.

(λA)ij:=λaij(\lambda A)_{ij} := \lambda a_{ij}

The set F n×p\mathbb{F}^{\,n\times p} with matrix addition and matrix scalar multiplication is a vector space over F.\mathbb{F}.

Let A∈F n×pA \in \mathbb{F}^{\,n\times p} and B∈F p×mB \in \mathbb{F}^{\,p\times m} be matrices such that AA has as many columns as BB has rows. Matrix multiplication is the matrix AB∈F n×mAB \in \mathbb{F}^{\,n\times m} whose element in row ii and column jj is the sum of the products of the elements of row ii of AA with the elements of column jj of B.B.

(AB)ij:=ai1b1j+ai2b2j+…+aipbpj=∑k=1paikbkj(AB)_{ij} := a_{i1}b_{1j}+a_{i2}b_{2j}+\ldots+a_{ip}b_{pj}=\sum_{k=1}^p a_{ik}b_{kj}

Matrix multiplication has the following properties:

  • Associativity: (AB)C=A(BC)  ∀ A∈F n×p,B∈F p×m,C∈F m×q.(AB)C=A(BC) \,\,\forall\,A \in \mathbb{F}^{\,n\times p},B \in \mathbb{F}^{\,p\times m},C \in \mathbb{F}^{\,m\times q}.
  • Distributivity: A(B+C)=AB+ACA(B+C)=AB+AC and (A+D)B=AB+DB(A+D)B=AB+DB ∀ A∈F n×p,B∈F p×m,C∈F p×m,D∈F n×p.\forall\,A \in \mathbb{F}^{\,n\times p},B \in \mathbb{F}^{\,p\times m},C \in \mathbb{F}^{\,p\times m}, D \in \mathbb{F}^{\,n\times p}.
  • Non-commutativity: in general AB≠BA,AB \neq BA, even when both products are defined.

Let n≥1.n \geq 1. The identity matrix In∈F n×nI_n \in \mathbb{F}^{\,n\times n} is the square matrix whose diagonal elements are 11 and whose other elements are 0.0. The identity matrix has the following property for every A∈F n×p:A\in\mathbb{F}^{\,n\times p}:

InA=AIp=AI_n A = AI_p = A

Let A∈F p×pA \in \mathbb{F}^{\,p\times p} be a square matrix and n∈N.n \in \mathbb{N}. The matrix power An∈F p×pA^n \in \mathbb{F}^{\,p\times p} is defined recursively by A0=IpA^0=I_p and, for n≥1,n\geq1,

An:=An−1A.A^n:=A^{n-1}A.

Let A∈F n×pA\in \mathbb{F}^{\,n\times p} be a matrix. The transpose of AA is the matrix A⊺∈F p×nA^\intercal\in \mathbb{F}^{\,p\times n} whose rows are the columns of A.A.

A⊺=[a11a21…an1a12a22…an2⋮⋮⋱⋮a1pa2p…anp]A^\intercal=\begin{bmatrix} a_{11} & a_{21} & \ldots & a_{n1} \\ a_{12} & a_{22} & \ldots & a_{n2} \\ \vdots & \vdots & \ddots & \vdots \\ a_{1p} & a_{2p} & \ldots & a_{np} \end{bmatrix}

The transpose has the following properties for A,B∈F n×pA,B \in \mathbb{F}^{\,n\times p} and C∈F p×m:C \in \mathbb{F}^{\,p\times m}:

(A⊺)⊺=A\left(A^\intercal\right)^\intercal=A (A+B)⊺=A⊺+B⊺\left(A+B\right)^\intercal = A^\intercal+B^\intercal (AC)⊺=C⊺A⊺\left(AC\right)^\intercal=C^\intercal A^\intercal

Let A∈F n×nA\in \mathbb{F}^{\,n\times n} be a square matrix. The trace of AA is the sum of its diagonal elements.

tr(A):=∑i=1naii∈F\mathrm{tr}(A):=\sum_{i=1}^n a_{ii} \in \mathbb{F}

The trace has the following properties:

tr(α)=α  ∀ α∈F\mathrm{tr}(\alpha) = \alpha \,\,\forall\, \alpha \in \mathbb{F} tr(A±B)=tr(A)±tr(B)  ∀ A,B∈F n×n\mathrm{tr}(A\pm B) = \mathrm{tr}(A)\pm\mathrm{tr}(B) \,\,\forall\, A,B \in \mathbb{F}^{\,n\times n} tr(αA)=α tr(A)  ∀ α∈F,A∈F n×n\mathrm{tr}(\alpha A) = \alpha\,\mathrm{tr}(A) \,\,\forall\, \alpha \in \mathbb{F}, A \in \mathbb{F}^{\,n\times n} tr(AB)=tr(BA)  ∀ A∈F n×p,B∈F p×n\mathrm{tr}(AB) = \mathrm{tr}(BA) \,\,\forall\, A \in \mathbb{F}^{\,n\times p}, B \in \mathbb{F}^{\,p\times n} ∑iu‾i⊺Au‾i=tr(AT)  ∀ u‾i∈Fn,A∈F n×n  where  T=∑iu‾iu‾i⊺\sum_i \underline{u}_{i}^\intercal A \underline{u}_i = \mathrm{tr}(AT) \,\,\forall\, \underline{u}_i \in \mathbb{F}^n, A \in \mathbb{F}^{\,n\times n} \,\,\mathrm{where} \,\, T=\sum_i \underline{u}_i \underline{u}_i^\intercal

Let A∈F n×pA \in \mathbb{F}^{\,n\times p} be a matrix. The column space of AA is the subspace of Fn\mathbb{F}^n spanned by the columns of A.A. The row space of AA is the subspace of F1×p\mathbb{F}^{1\times p} spanned by the rows of A.A. The column space and the row space of a matrix have equal dimension.

Let A∈F n×pA\in \mathbb{F}^{\,n\times p} be a matrix. The rank of A,A, written rank A,\mathrm{rank}\,A, is the dimension of the column space of A.A. The rank of AA equals the maximum number of linearly independent columns of A,A, which equals the maximum number of linearly independent rows of A.A. The rank of AA is at most min⁡{n,p}.\min\{n,p\}.

Let A∈F n×pA\in \mathbb{F}^{\,n\times p} be a matrix with rank r.r. The matrix AA is full rank if and only if r=min⁡{n,p}.r=\min\{n,p\}. The matrix AA is rank deficient if and only if r<min⁡{n,p},r<\min\{n,p\}, in which case its rank deficiency is min⁡{n,p}−r.\min\{n,p\}-r.

Let A∈F n×nA \in \mathbb{F}^{\,n\times n} be a square matrix. The matrix AA is a diagonal matrix if and only if every element of AA that is not a diagonal element is 0.0.

Let A∈F n×pA \in \mathbb{F}^{\,n\times p} be a matrix. The matrix AA is a null matrix if and only if every element of AA is 0.0.

Let A∈F n×nA \in \mathbb{F}^{\,n\times n} be a square matrix. The matrix AA is a symmetric matrix if and only if A⊺=A.A^\intercal=A. The matrix AA is a skew-symmetric matrix if and only if A⊺=−A.A^\intercal=-A.

Let A∈F n×nA \in \mathbb{F}^{\,n\times n} be a square matrix. The matrix AA is an upper triangular matrix if and only if every element below the diagonal is 0,0, that is aij=0  ∀ i>j.a_{ij} = 0\,\,\forall\,i>j. The matrix AA is a lower triangular matrix if and only if every element above the diagonal is 0,0, that is aij=0  ∀ i<j.a_{ij} = 0\,\,\forall\,i<j.

Let A∈F n×nA \in \mathbb{F}^{\,n\times n} be a square matrix. The matrix AA is invertible if and only if there exists a matrix A−1∈F n×nA^{-1} \in \mathbb{F}^{\,n\times n} such that AA−1=A−1A=In.AA^{-1}=A^{-1}A=I_n. The matrix A−1A^{-1} is the inverse of A.A. If an inverse exists then it is unique.

Let A∈F n×nA \in \mathbb{F}^{\,n\times n} be a square matrix. The matrix AA is singular if and only if it is not invertible.

Let A∈F n×nA \in \mathbb{F}^{\,n\times n} be a square matrix. The following conditions are equivalent, and each is equivalent to AA being invertible:

  • AA has full rank;
  • the columns of AA form a basis of Fn;\mathbb{F}^n;
  • the transpose A⊺A^\intercal is invertible;
  • the equation Av‾=0‾A\underline{v}=\underline{0} has only the solution v‾=0‾.\underline{v}=\underline{0}.

The matrix inverse has the following properties for invertible matrices A,B∈F n×nA,B\in\mathbb{F}^{\,n\times n} and non-zero scalars λ∈F:\lambda\in\mathbb{F}:

(λA)−1=λ−1A−1\left(\lambda A\right)^{-1}=\lambda^{-1}A^{-1} (AB)−1=B−1A−1\left(AB\right)^{-1}=B^{-1}A^{-1} (A⊺)−1=(A−1)⊺\left(A^\intercal\right)^{-1}=\left(A^{-1}\right)^\intercal

Let A∈R n×nA \in \mathbb{R}^{\,n\times n} be a real square matrix. The matrix AA is an orthogonal matrix if and only if the product of AA with its transpose is the identity matrix.

AA⊺=A⊺A=InAA^\intercal=A^\intercal A=I_n

Equivalently, AA is orthogonal if and only if its columns form an orthonormal set in Rn\mathbb{R}^n with respect to the inner product ⟨u‾,v‾⟩=u‾⊺v‾,\langle\underline{u},\underline{v}\rangle=\underline{u}^\intercal\underline{v}, if and only if its rows form an orthonormal set. Orthogonal matrices have the following properties:

  • the inverse is the transpose: A−1=A⊺;A^{-1}=A^\intercal;
  • the product ABAB of orthogonal matrices AA and BB is orthogonal.

Let A∈R n×pA \in \mathbb{R}^{\,n\times p} be a real matrix with n≠p.n\neq p. The matrix AA is a semi-orthogonal matrix if and only if either n>pn>p and the columns of AA form an orthonormal set, or n<pn<p and the rows of AA form an orthonormal set.

Let A∈C n×pA \in \mathbb{C}^{\,n\times p} be a complex matrix. The conjugate transpose of AA is the matrix A∗∈C p×nA^*\in \mathbb{C}^{\,p\times n} whose rows are the columns of AA with each element replaced by its complex conjugate.

A∗=[a11‾a21‾…an1‾a12‾a22‾…an2‾⋮⋮⋱⋮a1p‾a2p‾…anp‾]A^*=\begin{bmatrix} \overline{a_{11}} & \overline{a_{21}} & \ldots & \overline{a_{n1}} \\ \overline{a_{12}} & \overline{a_{22}} & \ldots & \overline{a_{n2}} \\ \vdots & \vdots & \ddots & \vdots \\ \overline{a_{1p}} & \overline{a_{2p}} & \ldots & \overline{a_{np}} \end{bmatrix}

Let A∈C n×nA \in \mathbb{C}^{\,n\times n} be a complex square matrix. The matrix AA is a Hermitian matrix if and only if A∗=A,A^*=A, that is aij=aji‾.a_{ij}=\overline{a_{ji}}. The diagonal elements of a Hermitian matrix are real. A real symmetric matrix is Hermitian.

Let A∈C n×nA \in \mathbb{C}^{\,n\times n} be a complex square matrix. The matrix AA is a unitary matrix if and only if the product of AA with its conjugate transpose is the identity matrix.

AA∗=A∗A=InAA^*=A^*A=I_n

A real orthogonal matrix is unitary.

Let A∈C n×nA \in \mathbb{C}^{\,n\times n} be a Hermitian matrix, so that v‾∗Av‾\underline{v}^*A\underline{v} is real for every v‾∈Cn.\underline{v} \in \mathbb{C}^n. The matrix AA is positive definite if and only if v‾∗Av‾>0  ∀ v‾∈Cn∖{0‾}.\underline{v}^*A\underline{v} > 0 \,\, \forall \, \underline{v} \in \mathbb{C}^n \setminus \{\underline{0}\}. The matrix AA is negative definite if and only if v‾∗Av‾<0  ∀ v‾∈Cn∖{0‾}.\underline{v}^*A\underline{v} < 0 \,\, \forall \, \underline{v} \in \mathbb{C}^n \setminus \{\underline{0}\}. The matrix AA is positive semi-definite if and only if v‾∗Av‾≥0  ∀ v‾∈Cn.\underline{v}^*A\underline{v} \geq 0 \,\, \forall \, \underline{v} \in \mathbb{C}^n. The matrix AA is negative semi-definite if and only if v‾∗Av‾≤0  ∀ v‾∈Cn.\underline{v}^*A\underline{v} \leq 0 \,\, \forall \, \underline{v} \in \mathbb{C}^n. For a real symmetric matrix the same definitions apply with v‾⊺Av‾\underline{v}^\intercal A\underline{v} for v‾∈Rn.\underline{v} \in \mathbb{R}^n.