## Description

1. Suppose you have the following data

Observation X1 X2 Class

1 3 1 +1

2 3 -1 +1

3 6 1 +1

4 6 -1 +1

5 1 0 -1

6 0 1 -1

7 0 -1 -1

8 -1 0 -1

(a) [6 points] Find the equation of the hyperplane (in terms of w) WITHOUT solving a quadratic

programming (QP) problem. Make a sketch of the problem (i.e., plot the data, unique hyperplane and corresponding dashed lines).

(b) [1 point] Calculate the margin.

(c) [2 points] Find the α’s of the SVM for classification (again WITHOUT solving a QP problem).

2. In this problem (based on Problem 7 on p. 371), you will use support vector classification in order

to predict whether a given car gets high or low gas mileage based on the Auto data set. You can

download the data set at http://www-bcf.usc.edu/~gareth/ISL/data.html. It can be helpful

to do the lab starting on p. 359 of the textbook first.

(a) [1 point] Create a class variable that takes on a “1” for cars with gas mileage above the

median, and a “-1” for cars with gas mileage below the median.

(b) [5 points] Fit a support vector classifier with linear kernel to the data with various values of

cost (i.e., the parameter C from class), in order to predict whether a car gets high or low gas

mileage. Report the cross-validation errors associated with different values of this parameter.

Comment on your results.

1

(c) [5 points] Now repeat (b), this time using SVMs with radial basis and polynomial kernels,

with different values of gamma, degree and cost. Comment on your results.

In class, I said the gaussian kernel had the following form: K(Xi

, Xj ) = exp(−kXi−Xjk

2

2

/h2

).

However, in this software package e1071, the radial basis function has the form K(Xi

, Xj ) =

exp(−γkXi − Xjk

2

2

). Consequently, γ = 1/h2

.

(d) [1 point] Which kernel function do you prefer and why?

3. (a) [6 points] Make an R program that solves the LS-SVM for nonlinear regression given the

parameters γ and bandwidth h for the following gaussian kernel (do not implemenent the

cross-validation procedure)

K(Xi

, Xj ) = 1

√

2π

exp

−

kXi − Xjk

2

2

2h

2

.

(b) [3 points] Try your program on the following function (i.e., experiment with some values of γ

and h)

X <- seq(0, 1, length.out = 200)
y <- (sin(2*pi*(x-0.5)))^2 + rnorm(200, 0, 0.2)
Describe the effect when the two tuning parameters γ and h change. Finally, try the values
γ = 9.4365 and h = 0.16006. Does this seems a good fit to you?
2