Description
1 HMM with tied mixtures (30 points)
(Adapted from Murphy Exercise 13.4) Consider an HMM where the observation model has the form:
p(Ot
|Xt = j, θ) = X
K
k=1
wjkN (Ot
|µk, Σk) ∀j ∈ {1, · · · , M}. (1)
In this model, we assume there are M types of hidden states. The observation for each hidden state is given
above; it is a mixture of K Gaussians. However, the Gaussians are shared between the states, and the state
influences the mixing weights but not the means and covariances. This is called semi-continuous HMM or
tied-mixture HMM.
1. (10 points) List all the parameters of this HMM model
2. (10 points) Derive the E step. What do we need to estimate in the E step?
3. (10 points) Derive the M step. How do we update the parameters of the model?
2 Programming Question (Action Classification Using RNN) (70 points + 10
bonus)
In this section, you will train recurrent neural networks (RNNs) to classify human actions. RNNs are designed handle sequential data.
For human action recognition, you will be using skeleton data that encodes the 3D locations of 25 body
joints. The data is collected by Kinect v2. There are 10 different action classes. There are 4000 training
sequences, 800 validation sequences, and 1000 test sequences. Each sequence has 15 frames, each frame is
a 75-dimension vector (the xyz positions of 25 joints).
You will first train a LSTM for action classification. Then try to improve the network architecture and
attach your results with the jupyter notebook. Also add the hyper-parameters explored.
The detail instructions and questions are in the jupyter notebook RNN ActionClassify.ipynb. In this file,
there are 4 ‘To-Do’ locations for you to fill. The score of each ‘To-Do’ is specified at the spot.
You need the following extra packages:
pip install h5py
pip install git+https://github.com/pytorch/tnt.git@master
1
3 What to submit?
3.1 Blackboard submission
For Question 1, please put everything in one single pdf file and submit it on Blackboard, please include
your name and student ID in the first page of the pdf file. For Question 2, submit the jupyter notebook files
RNN ActionClassify.ipynb with your answers filled at the To Do spots. Put the pdf file and your jupyter
notebook file in a folder named: SUBID FirstName LastName (e.g., 10947XXXX barrack obama). Zip this
folder and submit the zip file on Blackboard. Your submission must be a zip file, i.e, SUBID FirstName LastName.zip.
3.2 Kaggle submission
For Question 2, you must submit a .csv file to Kaggle competition site https://www.kaggle.com/
c/cse512springhw5 to get the Categorization Accuracy. Note: For this homework ID and Class are in
range [0, n − 1] and [0, k − 1]. A submission file should contain two columns: Id and Class. The file should
contain a header and have the following format.
ID, Class
0, 3
1, 0
… …
(2)
A sample submission file is available from the competition site and our handout. You MUST use your Stony
Brook CS email account to submit.
4 Cheating warnings
Don’t cheat. You must do the homework yourself, otherwise you won’t learn. You must use your SBU ID
as your file name for the competition. Do not fake your Stony Brook ID to bypass the submission limitation
per 24 hours. Doing so will be considered cheating.
2
