03. Bigger matrices

Bigger matrices

0.00 Just as a 2-by-2 matrix defines a transformation of the plane, an $m$ -by- $n$ matrix defines a transformation $\mathbf{R}^{n}\to\mathbf{R}^{m}$ . An $m$ -by- $n$ matrix is a rectangular array of numbers with $m$ rows and $n$ columns.

Example:

$\begin{pmatrix}a&b&c\\ d&e&f\end{pmatrix}$ is a 2-by-3 matrix.

1.55 The transformation $\mathbf{R}^{n}\to\mathbf{R}^{m}$ associated to an $m$ -by- $n$ matrix $M$ is the map $v\mapsto Mv$ where:

$v=\begin{pmatrix}x_{1}\\ x_{2}\\ \vdots\\ x_{n}\end{pmatrix}$
$M=\begin{pmatrix}M_{11}&M_{12}&\cdots&M_{1n}\\ M_{21}&M_{22}&\cdots&M_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ M_{m1}&M_{m2}&\cdots&M_{mn}\end{pmatrix}$
$M v$ is the vector whose $j$ th entry is obtained by multiplying the $j$ th row of $M$ into the column vector $v$ , that is $Mv=\begin{pmatrix}M_{11}x_{1}+M_{12}x_{2}+\cdots+M_{1n}x_{n}\\ M_{21}x_{1}+M_{22}x_{2}+\cdots+M_{2n}x_{n}\\ \vdots\\ M_{m1}x_{1}+M_{m2}x_{2}+\cdots+M_{mn}x_{n}\end{pmatrix}$

This vector $M v$ has height $m$ because there are $m$ rows of $M$ to multiply into the vector $v$ .

4.05 For example, $\begin{pmatrix}a&b&c\\ d&e&f\\ g&h&i\end{pmatrix}\begin{pmatrix}x\\ y\\ z\end{pmatrix}=\begin{pmatrix}ax+by+cz\\ dx+ey+fz\\ gx+hy+iz\end{pmatrix}$ shows how a 3-by-3 matrix eats a vector of height 3 and outputs a vector of height 3.

Example:

5.00 Take $M=\begin{pmatrix}\cos\theta&-\sin\theta&0\\ \sin\theta&\cos\theta&0\\ 0&0&1\end{pmatrix}$ . We get $Mv=\begin{pmatrix}x\cos\theta-y\sin\theta\\ x\sin\theta+y\cos\theta\\ z\end{pmatrix}.$ We see that this is a rotation of 3-dimensional space which fixes the $z$ -axis and rotates by $\theta$ in the $x y$ -plane. We call it a rotation by $\theta$ about the $z$ -axis.

Example:

8.00 Take $M=\begin{pmatrix}1&0&0\\ 0&1&0\end{pmatrix}$ . We need to feed $M$ a vector of height 3; it will output a vector of height 2. In other words, $M$ defines a transformation $\mathbf{R}^{3}\to\mathbf{R}^{2}$ . What is the transformation? $\begin{pmatrix}1&0&0\\ 0&1&0\end{pmatrix}\begin{pmatrix}x\\ y\\ z\end{pmatrix}=\begin{pmatrix}x\\ y\end{pmatrix}.$ This is the projection to the $x y$ -plane (which squishes the $z$ -axis to the origin).

Example:

9.45 Take $M=\begin{pmatrix}1&0\\ 0&1\\ 0&0\end{pmatrix}$ . This gives a map $\mathbf{R}^{2}\to\mathbf{R^{3}}$ : $\begin{pmatrix}1&0\\ 0&1\\ 0&0\end{pmatrix}\begin{pmatrix}x\\ y\end{pmatrix}=\begin{pmatrix}x\\ y\\ 0\end{pmatrix}$ This is the inclusion map of the 2-dimensional $x y$ -plane into 3-dimensional space (putting it at height zero).

11.30 These rectangular (nonsquare) matrices change the dimension of the space we're working with, e.g. map from a lower to a higher dimensional space or vice versa. You might wonder why we matrices which are bigger than 3-by-3, given that we live in a 3-dimensional universe. In fact:

the theory of special relativity treats space and time on an equal footing, and the Lorentz transformations, which describe all the weird relativistic effects like time dilation and length contraction, mix up space and time, and are given by 4-by-4 matrices.

in statistics, data is often represented as a vector of samples; the more samples you have, the bigger the dimension of the vector you need to encode them.

More examples

Example:

13.00 Take $M=\begin{pmatrix}1&1\\ 2&0\\ 0&1\end{pmatrix}$ . This defines a map $\mathbf{R}^{2}\to\mathbf{R}^{3}$ : $M\begin{pmatrix}x\\ y\end{pmatrix}=\begin{pmatrix}x+y\\ 2x\\ y\end{pmatrix}.$ What does this map "look like"? Its image (the set of points in 3d which have the form $M v$ for some $v\in\mathbf{R}^{2}$ ) is a plane. To visualise the plane, we and draw the images of the $x$ - and $y$ -axes in $\mathbf{R}^{2}$ :

14.55 The $x$ -axis (vectors of the form $\begin{pmatrix}x\\ 0\end{pmatrix}$ ) goes to the set of vectors $\begin{pmatrix}x\\ 2x\\ 0\end{pmatrix}$ .
16.12. The $y$ -axis (vectors of the form $\begin{pmatrix}0\\ y\end{pmatrix}$ ) goes to the set of vectors $\begin{pmatrix}y\\ 0\\ y\end{pmatrix}$ .

The image of $M$ is the unique plane containing these two lines.

Example:

17.55 Take $M=\begin{pmatrix}1&0-1\\ 0&1&-1\end{pmatrix}$ . This defines a map $\mathbf{R}^{3}\to\mathbf{R}^{2}$ : $\begin{pmatrix}1&0-1\\ 0&1&-1\end{pmatrix}\begin{pmatrix}x\\ y\\ z\end{pmatrix}=\begin{pmatrix}x-z\\ y-z\end{pmatrix}.$ What does this map look like? Let's imagine it's projecting from 3-dimensional space onto the $x y$ -plane (by including $\mathbf{R}^{2}$ into $\mathbf{R}^{3}$ as the $x y$ -plane). The points $\begin{pmatrix}x\\ y\\ 0\end{pmatrix}$ on the $x y$ -plane go to $\begin{pmatrix}x\\ y\end{pmatrix}$ (i.e. they stay where they are). The point $\begin{pmatrix}0\\ 0\\ 1\end{pmatrix}$ on the $z$ -axis goes to $\begin{pmatrix}-1\\ -1\end{pmatrix}$ . This means that everything is being projected onto the $x y$ -plane; the projection is along straight line rays which point in the $\begin{pmatrix}-1\\ -1\\ -1\end{pmatrix}$ -direction (because to get from $\begin{pmatrix}0\\ 0\\ 1\end{pmatrix}$ to $\begin{pmatrix}-1\\ -1\\ 0\end{pmatrix}$ you have to go backwards 1 in each of the $x$ , $y$ and $z$ directions. This map is therefore a projection.

Projection in (-1,-1,-1)-direction to xy-plane

22.40 This line along which we're projecting has a name: it's called the kernel of $M$ . More on this later.