Description
1. Consider the following matrix and vector:
A =
1 0
1 −1
0 1
, d =
−1
2
1
.
a) Find the solution wb to minw kd − Awk2.
b) Make a sketch of the geometry of this particular problem in R
3
, showing the
columns of A, the plane they span, the target vector d, the residual vector and
the solution db= Awb.
2. Recall the cereal calorie prediction problem discussed previously. Assume the data
matrix for this problem is
A =
25 0 1
20 1 2
40 1 6
.
Each column contains the grams/serving of carbohydrates, fat, and protein, and each
row corresponds to a different cereal (Frosted Flakes, Froot Loops, Grape-Nuts). The
total calories per serving for each cereal are
b =
110
110
210
.
You will find it helpful to solve the problems numerically. Relevant Python commands
include numpy.linalg.inv and numpy.linalg.matrix rank.
a) Write a small program that solves the system of equations Ax = b. Recall the
solution x gives the calories/gram of carbohydrate, fat, or protein. What is the
solution? The solution may not agree with the known calories/gram, which are 4
for carbs, 9 for fat and 4 for protein.
b) Now suppose that you use a more refined breakdown of carbohydrates, into total
carbohydrates, complex carbohydrates and sugars (simple carbs). This gives 5
features to predict calories (the three carb features + fat and protein). The grams
of these features in 5 different cereals is measured to obtain this data matrix
A =
25 15 10 0 1
20 12 8 1 2
40 30 10 1 6
30 15 15 0 3
35 20 15 2 4
.
Here the first column represents grams of total carbohydrates per serving, the second column complex carbohydrates, the third column sugars, the fourth column
fat, and the fifth column protein. The total calories in a serving of each cereal are
b =
104
97
193
132
174
.
Consider the problem Ax = b with 5 features.
i. Does an exact solution exist? Why or why not? Hint: The condition for an
exact solution was studied in the previous period activity.
ii. Does a unique solution exist? Why or why not? Hint: The condition for a
unique solution was studied in the previous period activity.
iii. Suppose you ignore (remove) the total carbohydrates per serving (first column
of A). Find a unique solution to the modified least-squares problem and the
resulting squared error.
3. Suppose the four-by-three matrix A = TWT where T =
t1 t2
and WT =
”
wT
1
wT
2
#
. Further, let t1 =
0.5
0.5
0.5
0.5
, t2 =
0.5
−0.5
−0.5
0.5
, and w1 =
1
1
1
and w2 =
1
−2
1
.
a) What is the rank of A?
b) What is the dimension of the subspace spanned by the columns of T ?
c) Is Q = ATA 0?
d) Suppose b =
2
1
1
2
. Does the least squares problem minx kb − Axk
2
2
have a
unique solution?
e) Suppose we force x to lie in the subspace spanned by w1 and w2, that is, we
constrain x = Wx˜ where x˜ is a two-by-one vector. Does the least squares problem
minx kb − Axk
2
2
have a unique solution for x˜? Find at least one solution. Note
that the numbers are chosen in this problem so you can easily do the calculations
on paper.
