Description
1. Let X denote an n × p matrix with each row an input vector and y denote an
ndimensional vector of the output in the training set. For fixed q ≥ 1, define
Bridgeλ
(β) = (y − Xβ)
T
(y − Xβ) + λ
X
p
j=1
βj

q
for λ > 0.
Denote the minimal value of the penalty function over the least
squares solution set by
t0 = min
β:XT Xβ=XT y
X
p
j=1
βj

q
.
(a) Show that Bridgeλ
(β) for λ > 0 is a convex function in β, which is strictly
convex for q > 1.
(b) Show that for q > 1 there is a unique minimizer, βˆ(λ), with Pp
j=1 βˆ
j (λ)
q ≤
t0.
(c) Show that for q = 1 there exists a minimizer and for all minimizers, βˆ(λ),
the penalty function takes the same value
s(λ) ,
X
p
j=1
βˆ
j (λ)
q ≤ t0.
Thus for q ≥ 1, s(λ) is well defined as a function of λ on the interval (0, ∞).
(d) Show that minimizing Bridgeλ
(β) is equivalent to minimizing
(y − Xβ)
T
(y − Xβ) subject to X
p
j=1
βj

q ≤ s(λ).
2. Exercise 1 (Section 6.8) of [ISL] (p. 259).
3. Exercise 3.16 of [ESL] (p. 96).
4. The prostate data described in Chapter 3 of [ESL] have been divided into a
training set of size 67 and a test set of size 30. Carry out the following analyses
on the training set:
(a) Bestsubset linear regression with k chosen by 5fold crossvalidation.
(b) Bestsubset linear regression with k chosen by BIC.
(c) Lasso regression with λ chosen by 5fold crossvalidation.
(d) Lasso regression with λ chosen by BIC.
(e) Principle component regression with q chosen by 5fold crossvalidation.
For each analysis, compute and plot the crossvalidation or BIC estimates of the
prediction error as the model complexity increases as in Figure 3.7. Report the
final estimated model as well as the test error and its standard error over the
test set as in Table 3.3. Briefly discuss your results.