Description
In this programming assignment, you will implement techniques to learn a linear regressor that
takes an input vector x of 10 dimensions and outputs a scalar y. Specifically, this assignment
involves learning the regression parameters Θ=(w,w0)
t
, where w is a 10×1 vector and w0 is a
scalar, given the training data D={x[m], y[m]}, m=1,2, …, 10,000. D is stored in a Mx11 matrix
(data.txt), where the first 10 columns represent input x and the last column represent y. You will
implement the following methods to learn the regression parameters Θ based on the mean
squared loss function.
a. Implement in Tensorflow the closed form gradient-based solution to solve Θ analytically.
Write down the equation used as well as the computed values for Θ.
b. Implement in Tensorflow the gradient descent method to solve for Θ iteratively using all data
in D. Initialize Θ to 0.01 for all elements of Θ. Try different learning rates between 0.001
an 0.005. Write down the estimated values for Θ as well as plot the loss function value
change as a function of iteration number t.
c. For graduate students taking this class at 6000 level, implement in Tensorflow the Stochastic
Gradient Descent method with a batch size set between to 20 and 10. Give the estimated
values for Θ, the selected batch size, and plot the loss function value as a function of
iteration number t.
Do not use Tensorflow’s existing gradient descent and stochastic gradient descent functions for
tasks b and c. Submit your Tensorflow code via LMS, along with the required outputs as
specified above.