TD 6

Index

• Theoretical Background
• Exercises

Theoretical Background

Basic Concepts

• The shape of a matrix is given by $(\#rows, \#columns)$. For example, $M = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ \end{bmatrix}$ has dimensions $(2,3)$. A vector $v = \begin{bmatrix} 1 \\ 2 \\ 3 \\ \end{bmatrix}$ can be thought of a matrix of shape $(3,1)$. Conversely, $M$ could be interpreted as a collection of three vectors, $M = (\begin{bmatrix} 1 \\ 4 \\ \end{bmatrix}, \begin{bmatrix} 2 \\ 5 \\ \end{bmatrix}, \begin{bmatrix} 3 \\ 6 \\ \end{bmatrix})$
• You can think of a vector $v$ of shape $(n,1)$ as a point in a n-dimensional space (the $n$ elements of the vectors just indicate the coordinates of this point). The whole space can be described by all the linear combinations possible between some basis vectors (this is called the linear span).
• A matrix $M$ describes a linear transformation of this space, so if you obtain a new vector $w$ by multiplying $w = Mv$, $w$ just represents where $v$ "ends up being placed" after the space has being transformed by $M$. Also, if you think of $M$ as a collection of vectors, these vectors represent where each of the basis vectors end up being placed.
• A square matrix is a matrix with shape $(k,k)$ (i.e. same number of rows and columns). A triangular matrix is any square matrix where all elements either above or below the diagonal are $0$. For example, $A = \begin{bmatrix} 2 & 3 & 4 \\ \color{yellow} 0 & 5 & 6 \\ \color{yellow} 0 & \color{yellow} 0 & 7 \\ \end{bmatrix}$ or $B = \begin{bmatrix} 10 & \color{yellow} 0 & \color{yellow} 0 \\ 11 & 12 & \color{yellow} 0 \\ 13 & 14 & 15 \\ \end{bmatrix}$
• The identity matrix $I$ is a matrix with the value $1$ along its diagonal and $0$ everywhere else. This matrix represents not transforming the space at all. For example, $I = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix}$
• For obtaining the transpose $M^T$ of a matrix $M$, just swap the rows and columns of $M$. For example, $M = \begin{bmatrix} \color{magenta} 1 & \color{magenta} 2 & \color{magenta} 3 \\ \color{yellow} 4 & \color{yellow} 5 & \color{yellow} 6 \\ \color{cyan} 7 & \color{cyan} 8 & \color{cyan} 9 \\ \end{bmatrix}, M^T = \begin{bmatrix} \color{magenta} 1 & \color{yellow} 4 & \color{cyan} 7 \\ \color{magenta} 2 & \color{yellow} 5 & \color{cyan} 8 \\ \color{magenta} 3 & \color{yellow} 6 & \color{cyan} 9 \\ \end{bmatrix}$. You can also visualize it as: $M = \begin{bmatrix} 1 & \color{magenta} 2 & \color{yellow} 3 \\ \color{magenta} 4 & 5 & \color{cyan} 6 \\ \color{yellow} 7 & \color{cyan} 8 & 9 \\ \end{bmatrix}, M^T = \begin{bmatrix} 1 & \color{magenta} 4 & \color{yellow} 7 \\ \color{magenta} 2 & 5 & \color{cyan} 8 \\ \color{yellow} 3 & \color{cyan} 6 & 9 \\ \end{bmatrix}$

About Matrix Multiplication

• A matrix multiplication $A=BC$ can be interpreted as transforming the space with $C$ and then applying the transformation described by $B$ (read it right to left, like when reading functions e.g. $B(C(v))$). This is equivalent to applying the transformation $A$ directly.
• Consider two arbitrary matrices $P$ and $Q$ with dimensions $(m,k)$ and $(k,n)$ respectively ($k,m,n$ are different integers). A new matrix $M$ with shape $(m,n)$ can be obtained by multiplying $P$ and $Q$ in the correct order, $M=PQ$, as the number of columns of $P$ matches the number of rows of $Q$. You can think of where this shape comes from as: $(\color{yellow} m, \color{magenta} k) \times (\color{magenta} k, \color{cyan} n) = (\color{yellow} m, \color{cyan} n)$.
• Conversely, the multiplication $QP$ isn't possible, as the shapes don't coincide: $(\color{magenta} k, \color{cyan} n) \times (\color{yellow} m, \color{magenta} k)$ because $n \neq m$.

Other Concepts

• Consider an arbitrary shape described by some points (vectors) in a n-dimensional space. The determinant of a square matrix $M$, $det(M)$ or $|M|$, represents how much the area (in the case of a 2-D space, the volume for a 3-D space, and so on) changes after performing the transformation described by $M$. This means that for $|M| > 1$, the area/volume of the arbitrary shape increases, for $|M| = 1$ it remains the same and for $0 < |M| < 1$ it shrinks. The same ideas apply when $|M| < 0$, with the added effect that the arbitrary shape gets reflected or reversed in some way. When $|M| = 0$, the whole space "collapses" into a lower dimension (for example, from 2D to 1D and all vectors ending up being placed into a single line).
• Given a matrix $M$, its inverse $M^{-1}$ represents reversing the transformation performed by $M$. If you apply $M$ and then undo it with $M^{-1}$, it's like no transformation was performed at all (i.e. identity matrix $I$). This is a way to interpret the fact that $MM^{-1}=M^{-1}M=I$.
• Consider the vectors in a linear space as arrows starting from the origin, with an "angle" with respect to the space's axes. When applying a linear transformation $M$ to the space, these arrows can get scaled in size and/or have their "angle" changed. Sometimes, a transformation might change the magnitude of the arrow described by $v$, scaling said magnitude by a value $\lambda$, while the arrow's "angle" remains unaffected. In this case, $v$ corresponds to an eigenvector and $\lambda$ to its respective eigenvalue. This is just a geometrical way to interpret the statement $Mv = \lambda v$. You can read it as:

  Applying the transformation $M$ to a vector $v$ is equivalent to just scaling the vector's size $v$ by some scalar value $\lambda$. Therefore, if $v$ is represented as an arrow, its original direction remains unaffected after the transformation.

• A rotation matrix describes a specific kind of transformation where all vectors are rotated around the origin. When a pure rotation is being performed, no stretching or squeezing should be present. Therefore, the determinant of a rotation matrix $R$ must be equal to $1$. Furthermore, the condition $R^T=R^{-1}$ must be fullfilled. You can rearrange this as $RR^T=I$, which is significantly easier to check.

Exercises

Exercise 1

Given these matrices $$ B = \begin{bmatrix} 2 & 3 & 4 \\ 1 & 5 & 6 \\ 7 & 8 & 9 \\ \end{bmatrix} C = \begin{bmatrix} 3 & 8 \\ 1 & 6 \\ \end{bmatrix} D = \begin{bmatrix} 4 & 1 \\ 9 & 2 \\ 3 & 7 \\ \end{bmatrix} $$ The matrix dimensions in this case are: $B \to (3,3)$, $C \to (2,2)$, $D \to (3,2)$. When multiplying two matrices, the first step is to check what shape the outcome matrix would be:

Exercise 3

$$\begin{aligned} &det \begin{pmatrix}\color{yellow}1&\color{magenta}2&\color{cyan}3\\\color{yellow}4&\color{magenta}5&\color{cyan}6\\\color{yellow}2&\color{magenta}4&\color{cyan}6\\\end{pmatrix} \\ =& + (\color{yellow}1) det \begin{pmatrix}\color{magenta}5&\color{cyan}6\\\color{magenta}4&\color{cyan}6\end{pmatrix} - (\color{magenta}2) det \begin{pmatrix}\color{yellow}4&\color{cyan}6\\\color{yellow}2&\color{cyan}6\end{pmatrix} + (\color{cyan}3) det \begin{pmatrix}\color{yellow}4&\color{magenta}5\\\color{yellow}2&\color{magenta}4\end{pmatrix} \\ =& (\color{yellow}1) (\color{magenta}5 \times \color{cyan}6 - \color{magenta}4 \times \color{cyan}6) - (\color{magenta}2) (\color{yellow}4 \times \color{cyan}6 - \color{yellow}2 \times \color{cyan}6) + (\color{cyan}3) (\color{yellow}4 \times \color{magenta}4 - \color{yellow}2 \times \color{magenta}5) \\ =& (\color{yellow}1) (6) - (\color{magenta}2) (12) + (\color{cyan}3) (6) \\ =& 6 - 24 + 18 = 0 \therefore \text{the matrix has no inverse} \\ \end{aligned}$$

Exercise 4

$$ R = \begin{bmatrix} \frac{\sqrt{3}}{2} & -\frac{1}{2} & 0 \\ \frac{\sqrt{2}}{4} & \frac{\sqrt{6}}{4} & -\frac{\sqrt{2}}{2} \\ \frac{\sqrt{2}}{4} & \frac{\sqrt{6}}{4} & \frac{\sqrt{2}}{2} \\ \end{bmatrix} $$ To confirm that $R$ is a rotation matrix, first verify that its determinant $|R|=1$. $$\begin{aligned} & \begin{vmatrix} \frac{\sqrt{3}}{2} & -\frac{1}{2} & 0 \\ \frac{\sqrt{2}}{4} & \frac{\sqrt{6}}{4} & -\frac{\sqrt{2}}{2} \\ \frac{\sqrt{2}}{4} & \frac{\sqrt{6}}{4} & \frac{\sqrt{2}}{2} \\ \end{vmatrix} \\\\ =& +(\frac{\sqrt{3}}{2}) \begin{vmatrix} \frac{\sqrt{6}}{4} & -\frac{\sqrt{2}}{2} \\ \frac{\sqrt{6}}{4} & \frac{\sqrt{2}}{2} \\ \end{vmatrix} - (-\frac{1}{2}) \begin{vmatrix} \frac{\sqrt{2}}{4} & -\frac{\sqrt{2}}{2} \\ \frac{\sqrt{2}}{4} & \frac{\sqrt{2}}{2} \\ \end{vmatrix} \\\\ =& \frac{\sqrt{3}}{2} (\frac{\sqrt{6}\sqrt{2}}{4 \times 2}+\frac{\sqrt{6}\sqrt{2}}{4 \times 2}) + \frac{1}{2} (\frac{\sqrt{2}\sqrt{2}}{4 \times 2} + \frac{\sqrt{2}\sqrt{2}}{4 \times 2}) \\ =& \frac{\sqrt{3}}{2} (\frac{\sqrt{2\times3\times2}}{4}) + \frac{1}{2} (\frac{2}{4}) \\ =& \frac{\sqrt{3}}{2} (\frac{2\sqrt{3}}{4}) + \frac{1}{2} (\frac{1}{2}) \\ =& \frac{3}{4} + \frac{1}{4} = 1 \\ \end{aligned}$$ Now, proceed to verify that $RR^T=I$. $$\begin{aligned} RR^T &= \begin{bmatrix} \frac{\sqrt{3}}{2} & -\frac{1}{2} & 0 \\ \frac{\sqrt{2}}{4} & \frac{\sqrt{6}}{4} & -\frac{\sqrt{2}}{2} \\ \frac{\sqrt{2}}{4} & \frac{\sqrt{6}}{4} & \frac{\sqrt{2}}{2} \\ \end{bmatrix} \begin{bmatrix} \frac{\sqrt{3}}{2} & \frac{\sqrt{2}}{4} & \frac{\sqrt{2}}{4} \\ -\frac{1}{2} & \frac{\sqrt{6}}{4} & \frac{\sqrt{6}}{4} \\ 0 & -\frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} \\ \end{bmatrix} \\\\ &= \begin{bmatrix} \frac{\sqrt{3}}{2}\frac{\sqrt{3}}{2} + \frac{1}{2}\frac{1}{2} & \frac{\sqrt{3}}{2}\frac{\sqrt{2}}{4} - \frac{1}{2}\frac{\sqrt{6}}{4} & \frac{\sqrt{3}}{2}\frac{\sqrt{2}}{4} - \frac{1}{2}\frac{\sqrt{6}}{4} \\ \frac{\sqrt{2}}{4}\frac{\sqrt{3}}{2} - \frac{\sqrt{6}}{4}\frac{1}{2} & \frac{\sqrt{2}}{4}\frac{\sqrt{2}}{4} + \frac{\sqrt{6}}{4}\frac{\sqrt{6}}{4} + \frac{\sqrt{2}}{2}\frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{4}\frac{\sqrt{2}}{4} + \frac{\sqrt{6}}{4}\frac{\sqrt{6}}{4} - \frac{\sqrt{2}}{2}\frac{\sqrt{2}}{2} \\ \frac{\sqrt{2}}{4}\frac{\sqrt{3}}{2} - \frac{\sqrt{6}}{4}\frac{1}{2} & \frac{\sqrt{2}}{4}\frac{\sqrt{2}}{4} + \frac{\sqrt{6}}{4}\frac{\sqrt{6}}{4} - \frac{\sqrt{2}}{2}\frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{4}\frac{\sqrt{2}}{4} + \frac{\sqrt{6}}{4}\frac{\sqrt{6}}{4} + \frac{\sqrt{2}}{2}\frac{\sqrt{2}}{2} \\ \end{bmatrix} \\\\ &= \begin{bmatrix} \frac{3}{4} + \frac{1}{4} & \frac{\sqrt{6}}{8} - \frac{\sqrt{6}}{8} & \frac{\sqrt{6}}{8} - \frac{\sqrt{6}}{8} \\ \frac{\sqrt{6}}{8} - \frac{\sqrt{6}}{8} & \frac{2}{16} + \frac{6}{16} + \frac{8}{16} & \frac{2}{16} + \frac{6}{16} - \frac{8}{16} \\ \frac{\sqrt{6}}{8} - \frac{\sqrt{6}}{8} & \frac{2}{16} + \frac{6}{16} - \frac{8}{16} & \frac{2}{16} + \frac{6}{16} + \frac{8}{16} \\ \end{bmatrix} \\\\ &= \begin{bmatrix}1&0&0\\0&1&0\\0&0&1\\\end{bmatrix} = I \\ \end{aligned}$$