Description
In this assignment, you will implement an SVM using the Matlab function quadprog. The most useful resource are the handwritten notes and the recitation on Oct. 26.
Part I: SVM via quadratic programming (20 points)
Examine the Matlab function quadprog. It can be used train an SVM (find the optimal w). Consider the input variables H,f,A,b,Aeq,beq,lb,ub. How should you set each of them so that the quadratic program that is solved is the solution to an SVM? How would you compute the weight vector w from the output of quadprog?
With these answers in hand, your next task is to implement an SVM using a call to quadprog with appropriately set input parameters. Write a function [y_pred] = svm_quadprog(X_train, Y_train, X_test, C) that “trains” an SVM i.e. obtains the solutions for the alpha weights using quadprog, then computes the weight vector w from these. It should also compute a prediction on the test set, using the test features and the weight vector.
The inputs to this function are:
- an NxD feature matrix X_train where N is the number of training instances and D is the feature dimension,
- an Nx1 label vector y_train for the training instances,
- an MxD feature matrix X_test where M is the number of test instances, and
- a scalar C, denoting the misclassification cost in an SVM.
The output is:
- an Mx1 predicted label vector y_pred for the test instances.
Part II: Testing your SVM (10 points)
Now test your SVM on the Pima Indians dataset from HW4. Reuse your KNN test script, but instead of testing different values of K or the bandwidth, test different values of C (e.g. 0.0001, 0.001, 0.01, 0.1, and 1). Name your adapted script svm_classification.m. Plot the performance of your SVM with the values of C on the x-axis, and the corresponding accuracies on the y-axis. Include the figure in a report file.
Important: Make sure to convert the ground-truth labels of 0 to -1. Then your predicted values should also be only -1 or 1.
Submission: Please include the following files:
- svm_quadprog.m
- svm_classification.m
- report.pdf
