These examples are carefully chosen to be rotation matrices. Note that if I gave you a random 3-by-3 matrix, it probably wouldn't be a rotation matrix, and isn't guaranteed to have any fixed vectors at all.
We'll analyse some examples of 3-by-3 rotation matrices, and then see to figure out the axis and angle of rotation for a general 3-by-3 rotation matrix.
Let . This is an example of a 3-by-3 rotation matrix. The top-left 2-by-2 block rotates the -plane, and the 1 in the bottom-right tells us that the -axis is fixed. More precisely: so the vectors in the -plane get rotated by the 2-by-2 rotation matrix , and the vector which points along the -axis is fixed.
A key point here is that the axis of rotation is fixed, i.e. any vector pointing in the -direction satisfies .
Let . This is another example of a 3-by-3 rotation matrix. What is the axis of rotation? We need to find a vector such that . In other words, This is three equations (one for each component): The equation is trivially satisfied. The other two equations imply . Therefore , and must point along the -axis.
By comparing with Example 1, the angle of rotation can be found by:
taking a vector which lives in the plane orthogonal to the axis,
applying to get ,
computing the angle between and using dot products.
For example, we could take (as so is orthogonal to the axis). Then , so . If is the angle between and then this implies , so degrees.
In fact, we can understand exactly what this rotation is doing by drawing the images of the basis vectors under (i.e. the three columns of ). The vector is fixed, goes to and goes to , so this rotates by 90 degrees about the -axis, sending the positive -axis to the positive -axis.
Take . This is another 3-by-3 rotation matrix; we'll find the axis and angle of rotation.
To find the axis , we need to solve : The first two equations imply , so the third is redundant, and any vector of the form is fixed. In other words, the axis points in the -direction.
To find the angle, pick a vector orthogonal to . For example, satisfies so is orthogonal to . We compute We also know that if is the angle between and then . Since , we get This tells us that is (or any of the other values that have ). Let's draw a picture to convince ourselves it's really (120 degrees).
The axis of rotation points out of the screen: