These examples are carefully chosen to be rotation matrices. Note that if I gave you a random 3by3 matrix, it probably wouldn't be a rotation matrix, and isn't guaranteed to have any fixed vectors at all.
12. Rotations
12. Rotations
We'll analyse some examples of 3by3 rotation matrices, and then see to figure out the axis and angle of rotation for a general 3by3 rotation matrix.
Example 1
Let A be the 3by3 matrix cos theta, minus sine theta, 0; sine theta, cos theta, 0; 0, 0, 1. This is an example of a 3by3 rotation matrix. The topleft 2by2 block rotates the x yplane, and the 1 in the bottomright tells us that the zaxis is fixed. More precisely: cos theta, minus sine theta, 0; sine theta, cos theta, 0; 0, 0, 1 times the vector x, y, z equals x cos theta minus y sine theta; x sine theta plus y cos theta; z so the vectors x, y, 0 in the x yplane get rotated by the 2by2 rotation matrix cos theta, minus sine theta; sine theta, cos theta, and the vector 0, 0, 1 which points along the zaxis is fixed.
A key point here is that the axis of rotation is fixed, i.e. any vector u pointing in the zdirection satisfies u = A u.
Example 2
Let B be the 3by3 matrix 0, 0, 1; 0, 1, 0; minus 1, 0, 0. This is another example of a 3by3 rotation matrix. What is the axis of rotation? We need to find a vector u equals x; y; z such that u = B u. In other words, x, y, z equals 0, 0, 1; 0, 1, 0; minus 1, 0, 0 times x, y, z, which equals z, y, minus x. This is three equations (one for each component): x equals z, y equals y and minus x equals z The equation y = y is trivially satisfied. The other two equations imply x = z = 0. Therefore u equals 0, y, 0, and u must point along the yaxis.
By comparing with Example 1, the angle of rotation can be found by:

taking a vector v which lives in the plane orthogonal to the axis,

applying B to get B v,

computing the angle between v and B v using dot products.
For example, we could take v equals 1, 0 ,0 (as v dot u equals 0 so v is orthogonal to the axis). Then B v equals 0, 0, minus 1, so v dot B v equals 0. If theta is the angle between v and B v then this implies cos theta = 0, so theta is plus or minus 90 degrees degrees.
In fact, we can understand exactly what this rotation is doing by drawing the images of the basis vectors e_1, e_2, e_3 under B (i.e. the three columns of B). The vector e_2 is fixed, e_3 goes to e_1 and e_1 goes to minus e_3, so this rotates by 90 degrees about the yaxis, sending the positive zaxis to the positive xaxis.
Example 3
Take C to be the 3by3 matrix 0, 0, 1; 1, 0, 0; 0, 1, 0. This is another 3by3 rotation matrix; we'll find the axis and angle of rotation.
To find the axis u equals x, y, z, we need to solve u = C u: x, y, z equals 0, 0, 1; 1, 0, 0; 0, 1, 0 times x, y, z, which equals z, x, y The first two equations imply x = y = z, so the third is redundant, and any vector of the form u equals x, x, x is fixed. In other words, the axis points in the 1, 1, 1direction.
To find the angle, pick a vector v orthogonal to u. For example, v equals 1, minus 1, 0 satisfies u dot v = 0 so is orthogonal to u. We compute v dot C v equals 1, minus 1, 0 dot 0, 1, minus 1, which equals minus 1. We also know that if theta is the angle between v and C v then minus 1 equals v dot C v equals norm v norm C v cos theta. Since norm v equals norm C v equals root 2, we get cos theta equals minus a half. This tells us that theta is 2 pi over 3 (or any of the other values that have cos theta equals minus a half). Let's draw a picture to convince ourselves it's really 2 pi over 3 (120 degrees).
The axis of rotation points out of the screen: