## Description

## Q1. Regularization

[15 points] We use polynomial regression for the prediction task of a dataset. The given dataset

includes a train set (train.csv) and a test set (test.csv). To illustrate the effect of regularization,

please first implement the following regression models using python language (third-party

packages are allowed).

Then, plot the data points of the train set and the regression lines of the

trained models. Finally, compute the RMSE of the trained models using the test set and make a

comparative discussion about underfitting and overfitting.

• Polynomial regression without regularization (polynomial to 5th power)

• L1 Regularized polynomial regression: 𝜆 = 1 and 𝜆 = 100

• L2 Regularized polynomial regression: 𝜆 = 1 and 𝜆 = 100

The given datasets can be downloaded at:

https://drive.google.com/drive/folders/1LSZNIEWf6XKnQtRw8L01tS6yAB67Aad2?usp=sharing

## Q2. Recommender System

Build up a collaborative filtering-based recommender system to provide effective hotel

recommendation. The training dataset as shown in the table below contains the ratings from 4 users

to 3 hotels. The ratings range from 1 point to 5 points.

Hotel 1 Hotel 2 Hotel 3

User 1 5 1 ?

User 2 4 ? 3

User 3 ? 4 5

User 4 3 3 4

We use the gradient descent algorithm to solve cost minimization in the collaborative filtering

model. Some settings are as follows.

• The constant learning rate 𝛼 = 0.0002

• The regularization parameter 𝜆 = 0.02

• The dimension for user/item feature vectors 𝐾 = 2

• The initial values for parameters 𝑥 = [

0.77 0.43 0.31

0.48 0.44 0.51] and 𝜃

𝑇 = [

0.19 0.62

0.68 0.78

0.18 0.08

0.36 0.92

]

a) [5 points] If we finally obtain 𝑥

(1) = [1.268 0.994]

𝑇

and 𝜃

(3) = [0.271 0.694]

𝑇

after the

training procedure, what is the rating of user 3 on hotel 1?

b) [10 points] Calculate the values of 𝑥1

(1)

(i.e., the first element in the item feature vector of

hotel 1) and 𝜃1

(2)

(i.e., the first element in the user feature vector of user 2) after the first

iteration.

c) [5 point] Implement the gradient descent algorithm to update the parameters 𝑥 and 𝜃 using

python language. Please calculate the ratings of user 2 on hotel 2 after 50 rounds and upload

the source code file.

ps. For a) and b), the detailed calculation process is required and the intermediate and final

results should be rounded to 3 decimal places.

## Q3. Neural Network

[10 points] Consider the following neural network:

Where 𝑎𝑖 = ∑ 𝑤𝑗

𝑖

𝑗 𝑧𝑗

𝑧𝑖 = 𝑓𝑖

(𝑎𝑖

) for 𝑖 = 1,2,3, 4 𝑧0 = 𝑎0

(an input neuron) 𝑓3

(𝑥) = relu(𝑥)

and 𝑓1

(𝑥) = 𝑓2

(𝑥) = 𝑓4

(𝑥) = sigmoid(𝑥). relu(𝑥) corresponds to a rectifier linear unit transfer

function defined as: relu(𝑥) = max {0,𝑥}.

The cost function is defined as 𝐽(𝑤) =

1

2

(𝑧4 − 𝑦)

2

.

(a) Write a function 𝐹 to simulate the neural network.

(b) Assume that we are given a training data 𝑥 = 1.0, 𝑦 = 0.1 what is the value of 𝜕𝐽

𝜕𝑤3

4?