Description
Introduction
Homework problems are selected from the course textbook: An Introduction to Statistical Learning.
Problem 1 (10 points)
Chapter 8, Exercise 4 (p. 332).
Problem 2 (10 points)
Chapter 8, Exercise 8 (p. 333).
Problem 3 (10 points)
Chapter 8, Exercise 10 (p. 334).
Problem 4 (10 points)
Chapter 8, Exercise 11 (p. 335).
Problem 5 (10 points)
Let xi
: i = 1, …, p be the input predictor values and a
(2s)
k
: k = 1, …, K be the K-dimensional output from
a 2-layer and M-hidden unit neural network with sigmoid activation σ(a) = {1 + e
−a}
−1
such that
a
(1s)
j = w
(1s)
j0 +
Xp
i=1
w
(1s)
ji xi
: j = 1, …, M
a
(2s)
k = w
(2s)
k0 +
X
M
j=1
w
(2s)
kj σ
a
(1s)
j
Show that there exists an equivalent network that computes exactly the same output values, but with
hidden unit activation functions given by tanh(a) = e
a−e
−a
e
a+e−a , i.e.
a
(1t)
j = w
(1t)
j0 +
Xp
i=1
w
(1t)
ji xi
: j = 1, …, M
a
(2t)
k = w
(2t)
k0 +
X
M
j=1
w
(2t)
kj tanh
a
(1t)
j
Hint: first derive the relation between σ(a) and tanh(a). Then show that the parameters of the two
networks differ by linear transformations.
1
