Description
1. Python and Numpy Warm-up Exercises This part of the homework is intended to help you
review your linear algebra and learn (or refresh your understanding of) how to implement linear
algebraic operations in Python using numpy (to which we refer in the code below as np). For each of
the problems below, write a method (e.g., problem1) that returns the answer for the corresponding
problem. Put all your methods in one file called homework1 WPIUSERNAME.py
(e.g., homework1 jrwhitehill.py). See the starter file homework1 template.py.
In all problems, you may assume that the the dimensions of the matrices and/or vectors are compatible
for the requested mathematical operations. Note: Throughout the assignment, please use np.array,
not np.matrix.
(a) Given matrices A and B, compute and return an expression for A + B. [ 1 pts ]
Answer (freebie!): While it is completely valid to use np.add(A, B), this is unnecessarily verbose; you really should make use of the “syntactic sugar” provided by Python’s/numpy’s operator
overloading and just write: A + B. Similarly, you should use the more compact (and arguably
more elegant) notation for the rest of the questions as well.
(b) Given matrices A, B, and C, compute and return AB − C (i.e., right-multiply matrix A by
matrix B, and then subtract C). Use dot or np.dot. [ 2 pts ]
(c) Given matrices A, B, and C, return A B+C>, where represents the element-wise (Hadamard)
product and > represents matrix transpose. In numpy, the element-wise product is obtained simply
with *. [ 2 pts ]
(d) Given column vectors x and y, compute the inner product of x and y (i.e., x
>y). [ 2 pts ]
(e) Given matrix A, return a matrix with the same dimensions as A but that contains all zeros. Use
np.zeros. [ 2 pts ]
(f) Given square matrix A and column vector x, use np.linalg.solve to compute A−1x. Do
not explicitly calculate the matrix inverse itself (e.g., np.linalg.inv, A ** -1) because this is
numerically unstable (and yes, it can sometimes make a big difference!). [ 2 pts ]
(g) Given square matrix A and row vector x, use np.linalg.solve to compute xA−1
. Hint: AB =
(B>A>)
>. [ 3 pts ]
(h) Given square matrix A and (scalar) α, compute A + αI, where I is the identity matrix with the
same dimensions as A. Use np.eye. [ 2 pts ]
(i) Given matrix A and integers i, j, return the jth column of the ith row of A, i.e., Aij . [ 1 pts ]
(j) Given matrix A and integer i, return the sum of all the entries in the ith row whose column index
is even, i.e., P
j:j is even Aij . Do not use a loop, which in Python can be very slow. Instead use
the np.sum function. [ 2 pts ]
(k) Given matrix A and scalars c, d, compute the arithmetic mean over all entries of A that are
between c and d (inclusive). In other words, if S = {(i, j) : c ≤ Aij ≤ d}, then compute
1
|S|
P
(i,j)∈S Aij . Use np.nonzero along with np.mean. [ 3 pts ]
(l) Given an (n×n) matrix A and integer k, return an (n×k) matrix containing the right-eigenvectors
of A corresponding to the k largest eigenvalues of A. Use np.linalg.eig. [ 3 pts ]
(m) Given a n-dimensional column vector x, an integer k, and positive scalars m, s, return an (n × k)
matrix, each of whose columns is a sample from multidimensional Gaussian distribution N (x +
mz, sI), where z is an n-dimensional column vector containing all ones and I is the identity matrix.
Use either np.random.multivariate normal or np.random.randn. [ 3 pts ]
(n) Given a matrix A with n rows, return a matrix that results from randomly permuting the
rows (but not the columns) in A. [ 2 pts]
1
2. Linear Regression via Analytical Solution
(a) Train an age regressor that analyzes a (48 × 48 = 2304)-pixel grayscale face image and outputs a real number ˆy that estimates how old the person is (in years). Your regressor should be
implemented using linear regression. The training and testing data are available here:
• https://s3.amazonaws.com/jrwprojects/age_regression_Xtr.npy
• https://s3.amazonaws.com/jrwprojects/age_regression_ytr.npy
• https://s3.amazonaws.com/jrwprojects/age_regression_Xte.npy
• https://s3.amazonaws.com/jrwprojects/age_regression_yte.npy
To get started, see the train age regressor function in homework1 template.py.
Note: you must complete this problem using only linear algebraic operations in numpy – you may
not use any off-the-shelf linear regression software, as that would defeat the purpose.
Compute the optimal weights w = (w1, . . . , w2304) for a linear regression model by deriving the
expression for the gradient of the cost function w.r.t. w and b, setting it to 0, and then solving.
Do not solve using gradient descent. The cost function is
fMSE(w) = 1
2n
Xn
i=1
(ˆy
(i) − y
(i)
)
2
where ˆy = g(x; w) = x
>w and n is the number of examples in the training set
Dtr = {(x
(1), y(1)), . . . ,(x
(n)
, y(n)
)}, each x
(i) ∈ R
2304 and each y
(i) ∈ R. Note that this simple
regression model does not include a bias term (which we will add later); correspondingly, please
do not include one in your own implementation. After optimizing w only on the training set,
compute and report the cost fMSE on the training set Dtr and (separately) on the testing set Dte.
[ 12 pts ]
3. Proofs For the proofs, please create a PDF (which you can generate using LaTex, or, if you prefer, a
scanned copy of your legible handwriting).
(a) Prove that
∇x
x
>a
= ∇x
a
>x
= a
for any two n-dimensional column vectors x, a. Hint: differentiate w.r.t. each element of x, and
then gather the partial derivatives into a column vector. [ 4 pts ]
(b) Prove that
∇x
x
>Ax
= (A + A>)x
for any n-dimensional column vector x and any n × n matrix A. [ 8 pts ]
(c) Based on the theorem above, prove that
∇x
x
>Ax
= 2Ax
for any n-dimensional column vector x and any symmetric n × n matrix A. [ 2 pts ]
(d) Based on the theorems above, prove that
∇x
h
(Ax + b)
>
(Ax + b)
i
= 2A> (Ax + b)
for any n-dimensional column vector x, any symmetric n × n matrix A, and any constant ndimensional column vector b. [ 4 pts ]
Submission: Create a Zip file containing both your Python and PDF files, and then submit on Canvas.
Teamwork: You may complete this homework assignment either individually or in teams up to 2 people