# Assignment #2: COMP4434 Big Data Analytics

\$30.00

Category:

5/5 - (1 vote)

## Question 1 [10 marks]

(a). [5 point] Consider using linear regression for binary classification on the label {0, 1}.
Here, we use a linear model
ℎ𝜃
(𝑥) = 𝜃1𝑥 + 𝜃0
and squared error loss 𝐿 =
1
2
(ℎ𝜃
(𝑥) − 𝑦)
2

. The threshold of the prediction is set as
0.5, which means the prediction result is 1 if ℎ𝜃
(𝑥) ≥ 0.5 and 0 if ℎ𝜃
(𝑥) < 0.5.

However, this loss has the problem that it penalizes confident correct predictions, i.e.,
ℎ𝜃
(𝑥) is larger than 1 or less than 0. Some students try to fix this problem by using an
absolute error loss 𝐿 = |ℎ𝜃
(𝑥) − 𝑦|. The question is: Will it fix the problem? Please
answer the question and explain it.

Furthermore, some other students try designing
another loss function as follows
𝐿 = {
max(0, ℎ𝜃
(𝑥)), 𝑦 = 0
⋯ , 𝑦 = 1
.

Although it is not complete yet, if it is correct in principle, please complete it and explain
how it can fix the problem. Otherwise, please explain the reason.

(b). [5 point] Consider the logistic regression model ℎ𝜃
(𝑥) = 𝑔(𝜃

𝑇𝑥), trained using the
binary cross entropy loss function, where 𝑔(𝑧) =
1
1+𝑒−𝑧
is the sigmoid function.

Some
students try modifying the original sigmoid function into the following one
𝑔(𝑧) =
𝑒
−𝑧
1+𝑒−𝑧
.

The model would still be trained using the binary cross entropy loss. How would the
model prediction rule, as well as the learnt model parameters 𝜃 , differ from
2

## Question 2 [20 marks]

Consider using logistic regression for classification problems. Four 3-dimensional data
points (𝑥1, 𝑥2, 𝑥3
)
𝑖
and the corresponding labels 𝑦
i
are given as follows.

Data point 𝑥1 𝑥2 𝑥3 y
D1 -0.120 0.300 -0.010 1
D2 0.200 -0.030 -0.350 -1
D3 -0.370 0.250 0.070 -1
D4 -0.100 0.140 -0.520 1

The learning rate 𝜂 is set as 0.2 and the initial parameter 𝜃[0] is set as [-0.09, 0, -0.19, –