Description
1. Let X =
1 1
1 0
0 1
.
a) Use the Gram-Schmidt orthogonalization procedure and hand calculation to find
an orthonormal basis for the space spanned by the columns of X. What geometric
object is described by the span of these bases?
b) Now interchange the columns of X, that is, define X˜ =
1 1
0 1
1 0
i. Do the columns of X span the same space as the columns of X˜ ?
ii. Use the Gram-Schmidt orthogonalization procedure to find an orthonormal
basis for the space spanned by the columns of X˜ . How does the geometric
object described by the span of this set of orthonormal bases compare to the
one in Part a?
iii. Are the bases vectors you found for X and X˜ the same? Does the space
spanned by the columns of a matrix depend on the order of the columns?
2. Let X =
1 1
1 0
0 1
as in the previous problem.
a) Place the orthonormal bases you found as columns of a matrix U.
b) Find UT U.
c) Since U contains a basis for space spanned by the columns of X you decide to
write each column of X as a linear combination of the columns of U: X =
U
a1 a2
. What is the dimension of a1? Briefly describe the meaning of a1
and a2.
d) Let A =
a1 a2
so that X = UA. Multiply both sides of this equation by
UT and solve for A.
3. Let the columns of an n-by-p (n > p) matrix X be linearly independent and U be an
orthonormal basis for the p-dimensional space spanned by the columns of X.
a) It can be shown that X = UT where T is a p-by-p invertible matrix. Briefly
explain why T should be invertible without resorting to a mathematical proof.
That is, explain why this result is intuitively reasonable.
b) Use the result in the previous item to show that the projection onto the space
spanned by X is identical to that onto the space spanned by U. That is, show
Px = X(XTX)
−1XT = PU = U(UT U)
−1UT
. Hint: Recall that (AB)
−1 =
B−1A−1
.
c) Express PU without a matrix inverse.
4. Consider the matrix and vector
X =
1 1
1 0
0 1
and b =
1
2
1
.
Note that X is defined identically in the preceding problems.
a) Make a sketch of the orthonormal bases U and the columns of X in three dimensions.
b) Use U and the result of the previous problem to compute the LS estimate ˆb =
X(XTX)
−1XT b.
5. Let z =
”
1
1
#
and define Q = zzT
.
a) Sketch the surface y = x
TQx where x =
”
x1
x2
#
. If you find 3-D sketching too
difficult, you may draw a contour map with labeled contours.
b) Let w =
”
1
−1
#
. Sketch the subspace spanned by z and the subspace spanned
by w on your sketch of the surface y = x
TQx.
c) Does the problem minx x
TQx have a unique solution?
d) Is Q 0? Is Q 0?
