38. Linear maps

Two definitions of linearity

0.00 At the outset of this course, we talked about the geometric transformations coming from matrices (rotations, reflections, shears etc). These geometric transformations have a name: they are called linear maps. In this video we'll give two definitions of linear maps and show they're equivalent. The first definition encapsulates how we've been dealing with linear maps so far:

Definition:

0.35 A map $f\colon\mathbf{R}^{n}\to\mathbf{R}^{m}$ is linear if there exists an $m$ -by- $n$ matrix $A$ such that $f(v)=Av$ for all $v\in\mathbf{R}^{n}$ .

1.16 However, some linear maps are more natural to describe in another way, without giving the matrix $A$ .

Fix $n$ . Consider the space of polynomials $P(x)$ of degree at most $n$ . Differentiation gives a map $\frac{d}{dx}\colon P\mapsto dP/dx$ from this space to itself. This map is linear. To understand why, we need to understand polynomials as vectors. We encode a polynomial $P(x)=a_{n}x^{n}+\cdots+a_{1}x+a_{0}$ as its vector of coefficients $\begin{pmatrix}a_{0}\\ a_{1}\\ \vdots\\ a_{n}\end{pmatrix}$ . Then $\frac{dP}{dx}=na_{n}x^{n-1}+\cdots+2a_{2}x+a_{1},$ 2.52 which corresponds to the vector of coefficients $\begin{pmatrix}a_{1}\\ 2a_{2}\\ \vdots\\ na_{n}\\ 0\end{pmatrix}$ . This is the same as $\begin{pmatrix}0&1&0&\cdots&0&0\\ 0&0&2&0&\cdots&0\\ 0&0&0&3&0&\vdots\\ \vdots&\ddots&\ddots&\ddots&\ddots&\vdots\\ 0&0&0&\cdots&0&n\\ 0&0&0&\cdots&0&0\end{pmatrix}\begin{pmatrix}a_{0}\\ a_{1}\\ \vdots\\ a_{n}\end{pmatrix}$ 4.02 so differentiation is linear (given by a matrix). In other words, if $v_{P}$ is the vector of coefficients of the polynomial $P$ then $Dv_{P}=v_{dP/dx}$ , where $D$ is this matrix.

4.30 This way of encoding polynomials as vectors is a bit artificial. For example, I could have chosen to write the vector with $a_{n}$ at the top and $a_{0}$ at the bottom, and the matrix $D$ would have ended up looking quite different. The fact that differentiation of polynomials is a linear map is an intrinsic fact about differentiation, and our proof above obscures that. So here's an equivalent definition of linearity which is more intrinsic.

Definition:

5.00 A map $f$ is linear if:

$f(v+w)=f(v)+f(w)$ for all $v, w$
$f(\lambda v)=\lambda f(v)$ for all $v$ and for all $\lambda\in\mathbf{R}$ .

I haven't specified the domain and target of $f$ because I want to be intentionally vague: this definition makes sense whenever the domain and target of $f$ admit operations of addition and rescaling (e.g. spaces of polynomials or functions as well as just $\mathbf{R}^{n}$ ). In the final video of the course, we'll see that the natural setting for this definition is the setting of vector spaces.

6.15 Differentiation of polynomials is linear because $\frac{d}{dx}(P+Q)=\frac{dP}{dx}+\frac{dQ}{dx}$ and $\frac{d(\lambda P)}{dx}=\lambda\frac{dP}{dx}$ for any constant $\lambda$ and polynomials $P(x),Q(x)$ .

6.47 The function $f\colon\mathbf{R}\to\mathbf{R}$ which converts metres to feet is linear. Since 1 metre is $\approx 3.281$ feet, $f(x)=3.281x$ . If you double the number of metres, you double the number of feet. If you take two distances $x$ metres and $y$ metres you can add them and then convert to feet ( $f(x+y)$ ) or you can convert and then add ( $f(x)+f(y)$ ) and you get the same answer. So $f$ is linear.

8.26 The function $f\colon\mathbf{R}\to\mathbf{R}$ which converts Celsius to Kelvin is not linear. Recall that $f(0)\approx 273$ . Any linear map satisfies $f(0)=0$ , because $f(0)=f(00)=0f(0)=0$ (some of those $0$ s are numbers, some are vectors!).

9.41 I'm told the way they used to mark exams in Oxford was to take the marks from each question, square them and add them up. For example, if there were two questions and you got marks $x$ and $y$ then your final score would be $x^{2}+y^{2}$ . This rewards those who do very well on a couple of questions (instead of scatter-gunning a little bit of knowledge over all questions). This function $f(x,y)=x^{2}+y^{2}$ is not a linear map! For example, if you score $1$ and $0$ then you get $1$ in total, but if you double your score for $x$ then you quadruple your total. Sadly for those taking the exam, $f(0,0)=0$ .

Equivalence

Lemma:

11.36 These two definitions of linearity are equivalent. In other words, the conditions

$f(v+w)=f(v)+f(w)$ and $f(\lambda v)=\lambda f(v)$

imply there exists a matrix $A$ such that $f(v)=Av$ , and any map of this form satisfies these conditions.

12.26 If $f(v)=Av$ for some matrix $A$ then $f(v+w)=A(v+w)=Av+Aw=f(v)+f(w)$ and $f(\lambda v)=A(\lambda v)=\lambda Av=\lambda f(v).$

13.05 Conversely, consider the basis vectors $e_{1}=\begin{pmatrix}1\\ 0\\ \vdots\\ 0\end{pmatrix},\qquad e_{2}=\begin{pmatrix}0\\ 1\\ \vdots\\ 0\end{pmatrix},\qquad\cdots,\qquad e_{n}=\begin{pmatrix}0\\ \vdots\\ 0\\ 1\end{pmatrix}.$ Let $A$ be the matrix whose columns are $f(e_{1}),f(e_{2}),\ldots,f(e_{n})$ .

14.11 Then $f(v)=f\begin{pmatrix}v_{1}\\ \vdots\\ v_{n}\end{pmatrix}=f(v_{1}e_{1}+\cdots+v_{n}e_{n})=v_{1}f(e_{1})+\cdots+v_{n}f% (e_{n}).$ Since $f(e_{k})$ is the $k$ th column of $A$ , it agrees with $Ae_{k}$ (also the $k$ th column of $A$ ). Therefore $f(v)=v_{1}Ae_{1}+\cdots+v_{n}Ae_{n}=A(v_{1}e_{1}+\cdots+v_{n}e_{n})=Av,$ which shows that $f$ is linear in the sense that it has the form $f(v)=Av$ for some matrix $A$ .