## Description

1. Consider the following matrix and vector:

A =

1 0

1 −1

0 1

, d =

−1

2

1

.

a) Find the solution wb to minw kd − Awk2.

b) Make a sketch of the geometry of this particular problem in R

3

, showing the

columns of A, the plane they span, the target vector d, the residual vector and

the solution db= Awb.

2. Recall the cereal calorie prediction problem discussed previously. Assume the data

matrix for this problem is

A =

25 0 1

20 1 2

40 1 6

.

Each column contains the grams/serving of carbohydrates, fat, and protein, and each

row corresponds to a different cereal (Frosted Flakes, Froot Loops, Grape-Nuts). The

total calories per serving for each cereal are

b =

110

110

210

.

You will find it helpful to solve the problems numerically. Relevant Python commands

include numpy.linalg.inv and numpy.linalg.matrix rank.

a) Write a small program that solves the system of equations Ax = b. Recall the

solution x gives the calories/gram of carbohydrate, fat, or protein. What is the

solution? The solution may not agree with the known calories/gram, which are 4

for carbs, 9 for fat and 4 for protein.

b) Now suppose that you use a more refined breakdown of carbohydrates, into total

carbohydrates, complex carbohydrates and sugars (simple carbs). This gives 5

features to predict calories (the three carb features + fat and protein). The grams

of these features in 5 different cereals is measured to obtain this data matrix

A =

25 15 10 0 1

20 12 8 1 2

40 30 10 1 6

30 15 15 0 3

35 20 15 2 4

.

Here the first column represents grams of total carbohydrates per serving, the second column complex carbohydrates, the third column sugars, the fourth column

fat, and the fifth column protein. The total calories in a serving of each cereal are

b =

104

97

193

132

174

.

Consider the problem Ax = b with 5 features.

i. Does an exact solution exist? Why or why not? Hint: The condition for an

exact solution was studied in the previous period activity.

ii. Does a unique solution exist? Why or why not? Hint: The condition for a

unique solution was studied in the previous period activity.

iii. Suppose you ignore (remove) the total carbohydrates per serving (first column

of A). Find a unique solution to the modified least-squares problem and the

resulting squared error.

3. Suppose the four-by-three matrix A = TWT where T =

t1 t2

and WT =

”

wT

1

wT

2

#

. Further, let t1 =

0.5

0.5

0.5

0.5

, t2 =

0.5

−0.5

−0.5

0.5

, and w1 =

1

1

1

and w2 =

1

−2

1

.

a) What is the rank of A?

b) What is the dimension of the subspace spanned by the columns of T ?

c) Is Q = ATA 0?

d) Suppose b =

2

1

1

2

. Does the least squares problem minx kb − Axk

2

2

have a

unique solution?

e) Suppose we force x to lie in the subspace spanned by w1 and w2, that is, we

constrain x = Wx˜ where x˜ is a two-by-one vector. Does the least squares problem

minx kb − Axk

2

2

have a unique solution for x˜? Find at least one solution. Note

that the numbers are chosen in this problem so you can easily do the calculations

on paper.