Description
Part A: Theory (20pts)
(15pts) Support Vector Machines (Revisited)
We now look at a different type of SVM that is designed for domain adaptation and optimizes
the hyperplanes given by wS (source hyperplane) before optimizing wT (target hyperplane). The
process begins by training a support vector machine on source data then once data from the target
are available, train a new SVM using the hyperplane from the first SVM and the data from the
target to solve for a new “domain adaptation” SVM.
The primal optimization problem is given by
min
wT ,ξ
1
2
∥wT ∥
2 +C
n
∑
i=1
ξi −Bw
T
TwS
s.t. yi(w
T
T
xi +b) ≥ 1−ξi ∀i ∈ {1,…,n}
ξi ≥ 0 ∀i ∈ {1,…,n}
where wS is hyperplane trained on the source data (assumed to be known), wT is hyperplane for
the target, yi ∈ {±1} is the label for instance xi
, C & B are regularization parameters defined by
the user and ξi
is a slack variable for instance xi
. The problem becomes finding a hyperplane,
wT , than minimizes the above objective function subject to the constraints. Solve/derive the dual
optimization problem.
(5pts) AdaBoost: How much do you understand the algorithm?
In a couple paragraphs, answer: Why does boosting (generally) work?
Part B: Practice (20pts)
You are free to use functions already implemented in Matlab, Python or R with the exception of
problem 1. I recommend using Python’s Scikit-learn (http://scikit-learn.org/stable/)
as is implements most of the methods we will be discussing in this course. . . as well as problems
in this homework!
(15pts) Support Vector Machines (Revisited)
Implement the domain adaptation SVM from the first problem in the theory section. A data set
for the source and target domains (both training and testing) have been uploaded to D2L. There
are several ways to implement this algorithm. If I were doing this for an assignment, I would
implement the SVM directly using quadratic programming. For example, see CVX (Python and
Matlab: https://goo.gl/3f7StQ), but there are other tools available.
(5pts) An Experiment with Ensembles
Perform an experiment that evaluates the impact of the size of an ensemble on the testing error
of the ionosphere.csv data set. Sweep the ensemble size from 2 to whatever you want to
arizona.edu 2 March 10, 2017
Spring 2017 Dept. of ECE University of Arizona
see of the testing error converges as the ensemble size is increased. Perform this experiment with
Bagging and AdaBoost. Plot the testing error for both Bagging and AdaBoost in the same plot.
arizona.edu 3 March 10, 2017
