Description
1. Feed Forward Neural Network Implementation. Implement a Feed Forward Neural Network
in Python, with an input layer with S1 units, one hidden layer with S2 units, and an output layer with
S3 units using the backpropagation algorithm. The network will be trained using data {(xi
, yi
)}
N
i=1
for xi ∈ R
S1 and yi ∈ R
S3
. The code must allow specifying the following activation functions:
sigmoid, hyperbolic tangent and rectifier linear activation. The code must output all the learned
weights and biasses of all layers as well as the activations of the last layer.
2. Auto-Encoder Implementation. Implement an auto-encoder neural network in Python, by
modifying the code in question 1. The network receives the input data {xi}
N
i=1. The code must
allow specifying the following activation functions: sigmoid, hyperbolic tangent and rectifier linear
activation. The code must output all the learned weights and biasses of all layers as well as the
activations of the last layer.
3. Testing Algorithms on Data.
Part A. Take the provided dataset, which consists of face images from 10 individuals corresponding to 10 classes.
a) Train the NN in the question 1 to separate the training face images in 10 classes. Report the
classification error on the test data as a function of the number of neurons/units in the hidden layer.
Report the results for all three activation functions. How does the result depend on activation function and the number of hidden neurons? Fully explain.
b) Train 1 versus all SVM classifier to separate the training face images in 10 classes. Report the
classification error on the test data.
c) Train 1 versus all logistic regression classifier to separate the training face images in 10 classes.
Report the classification error on the test data.
d) Repeat parts i, ii, iii by applying PCA to data first to reduce the dimension to d = 100 and then
learning a classifier. How does the result change with respect to using the original data.
e) Repeat parts i, ii, iii by applying Auto Encode to data first to reduce the dimension to d = 100
and then learning a classifier. How does the result change with respect to using PCA and using the
original data.
f) Visualized the data by applying PCA to data and reduce the dimension of data to d = 2. Repre1
sent different classes by a different marker and color in your plot. Are the data separated according
to classes? Explain.
g) Apply Kmeans to the original dataset with K = 10. Show the results by plotting the 2-
dimensional data and indicating data in each cluster by a different color. Report the clustering
error on the dataset. Explain why Kmeans is or is not successful in recovering the true clustering/grouping of the data. Repeat this part by applying Kmeans to d = 100 dimensional PCA
representations.
h) Apply Spectral Clustering with Gaussian RBF kernels to the original data. For spectral clustering, connect each point to its K nearest neighbors using Euclidean distance between points and for
any two points i and j connected, set the weights to be wij = e
−kyi−yjk
2
2
/(2σ
2
)
. Try several values
of K and σ and report the clustering error on the dataset as a function of K and σ. Explain why
Spectral Clustering is or is not successful in recovering the true clustering/grouping of the data.
How does spectral clustering perform compared to kmeans?
Homework Submission Instructions:
–Submission: You must submit all your plots and your Python code (.py file) via email, BY THE
DEADLINE. To submit, please send an email to me and CC both TAs.
– The title of your email must be “CS6140: HW4: Your First and Last Name”.
– You must attach a single zip file to your email that contains all python codes and plots and a
readme file on how to run your files.
– The name of the zip file must be “HW4: Your First and Last Name”.
2
