Inner Product Spaces

Let $\mathcal{V}$ be a vector space over the field $\mathbb{F}.$ A bilinear form on $\mathcal{V}$ is a function from pairs of vectors in $\mathcal{V}$ to $\mathbb{F}$ written $⟨\cdot ,\cdot ⟩:\mathcal{V}\times \mathcal{V}\to \mathbb{F}$ and satisfies the bilinear form axioms:

$$⟨\lambda {\underset{―}{v}}_1+\mu {\underset{―}{v}}_2,{\underset{―}{v}}_3⟩=\lambda ⟨{\underset{―}{v}}_1,{\underset{―}{v}}_3⟩+\mu ⟨{\underset{―}{v}}_2,{\underset{―}{v}}_3⟩\mathrm{\forall }{\underset{―}{v}}_1,{\underset{―}{v}}_2,{\underset{―}{v}}_3\in \mathcal{V},\mathrm{\forall }\lambda ,\mu \in \mathbb{F}$$ $$⟨{\underset{―}{v}}_1,\lambda {\underset{―}{v}}_2+\mu {\underset{―}{v}}_3⟩=\lambda ⟨{\underset{―}{v}}_1,{\underset{―}{v}}_2⟩+\mu ⟨{\underset{―}{v}}_1,{\underset{―}{v}}_3⟩\mathrm{\forall }{\underset{―}{v}}_1,{\underset{―}{v}}_2,{\underset{―}{v}}_3\in \mathcal{V},\mathrm{\forall }\lambda ,\mu \in \mathbb{F}.$$

A bilinear form is conjugate symmetric if $⟨{\underset{―}{v}}_1,{\underset{―}{v}}_2⟩=\overline{⟨{\underset{―}{v}}_2,{\underset{―}{v}}_1⟩}\mathrm{\forall }{\underset{―}{v}}_1,{\underset{―}{v}}_2\in \mathcal{V}$ where for a complex number $z=a+ib\in \mathbb{C}$ the complex conjugate is $\bar{z}=a-ib\in \mathbb{C}.$ If $\mathbb{F}=\mathbb{R}$ conjugate symmetry is simply symmetry: $⟨{\underset{―}{v}}_1,{\underset{―}{v}}_2⟩=⟨{\underset{―}{v}}_2,{\underset{―}{v}}_1⟩.$

A bilinear form is positive definite if $⟨\underset{―}{v},\underset{―}{v}⟩>0\mathrm{\forall }\underset{―}{v}\in \mathcal{V}\setminus \{\underset{―}{0}\},$ negative definite if $⟨\underset{―}{v},\underset{―}{v}⟩<0\mathrm{\forall }\underset{―}{v}\in \mathcal{V}\setminus \{\underset{―}{0}\},$ positive semi-definite if $⟨\underset{―}{v},\underset{―}{v}⟩\ge 0\mathrm{\forall }\underset{―}{v}\in \mathcal{V}\setminus \{\underset{―}{0}\}$ and negative semi-definite if $⟨\underset{―}{v},\underset{―}{v}⟩\le 0\mathrm{\forall }\underset{―}{v}\in \mathcal{V}\setminus \{\underset{―}{0}\}.$

An inner product is a positive definite, conjugate symmetric bilinear form on $\mathcal{V}.$ A vector space is an inner product space if it is equipped with the inner product.

Let $\mathcal{V}$ be an inner product space over the field $\mathbb{F}$ and $\underset{―}{v}\in \mathcal{V}.$ The norm or length $\Vert \underset{―}{v}\Vert$ of $\underset{―}{v}$ is the square root of the inner product with itself:

$$\Vert \underset{―}{v}\Vert :=\sqrt{⟨\underset{―}{v},\underset{―}{v}⟩}\in \mathbb{R}$$

The angle $\theta$ between two vectors $\underset{―}{u},\underset{―}{v}\in \mathcal{V}$ is defined as the inverse cosine of the quotient of the inner product of the two vectors and the product of their norms:

$$\theta :={\cos }^{-1}\left(\frac{⟨\underset{―}{u},\underset{―}{v}⟩}{\Vert \underset{―}{u}\Vert \Vert \underset{―}{v}\Vert }\right)\in [0,\pi ]$$

Two vectors $\underset{―}{u},\underset{―}{v}\in \mathcal{V}$ are orthogonal to each other $\underset{―}{u}\perp \underset{―}{v}$ if $⟨\underset{―}{u},\underset{―}{v}⟩=0.$ If $\underset{―}{u}$ and $\underset{―}{v}$ are unit vectors $\Vert \underset{―}{u}\Vert =\Vert \underset{―}{v}\Vert =1$ then they are orthonormal.

For a subset of vectors $U\subseteq \mathcal{V},$ they are said to be an orthogonal set or orthonormal set if all vectors contained within the sets are pairwise orthogonal or orthonormal respectively. Orthonormal sets are linearly independent and a set of $n$ orthonormal vectors in an $n$-dimensional vector space is a basis.

The Gram-Schmidt orthonormalisation procedure is a method to orthonormalise a set of linearly independent vectors ${\underset{―}{u}}_1,\dots {\underset{―}{u}}_n\in \mathcal{U}$ in an inner product space $\mathcal{U}.$ The resultant orthonormalised vectors ${\underset{―}{v}}_1,\dots {\underset{―}{v}}_n$ will span $\mathcal{U}.$

$$\begin{array}{cccccc}{w}_1& :=& u_1& v_1& :=& \frac{w_1}{\Vert w_1\Vert }\\ w_2& :=& u_2-(u_2\cdot v_1)v_1& v_2& :=& \frac{w_2}{\Vert w_2\Vert }\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ w_n& :=& u_n-\sum _{i=1}^{n-1}(u_n\cdot v_i)v_i& v_n& :=& \frac{w_n}{\Vert w_n\Vert }\end{array}$$

The Cauchy-Schwarz inequality states that for an inner product space $\mathcal{V}$ and ${\underset{―}{v}}_1,{\underset{―}{v}}_2\in \mathcal{V}:$

$$|⟨{\underset{―}{v}}_1,{\underset{―}{v}}_2⟩|\le \Vert {\underset{―}{v}}_1\Vert \Vert {\underset{―}{v}}_2\Vert$$

with equality if and only if ${\underset{―}{v}}_1$ and ${\underset{―}{v}}_2$ are linearly dependent.