Description
What to Turn in
• Your code and a Readme file for compiling and executing your code.
• A detailed write up that reports the accuracy, precision, recall and F1 score obtained on the
three test sets for the following algorithms and feature types:
– Multinomial Naive Bayes on the Bag of words model
– Discrete Naive Bayes on the Bernoulli model
– Logistic Regression on both Bag of words and Bernoulli models
– SGDClassifier on both Bag of words and Bernoulli models
Your report should also describe how you tuned the hyper-parameters for logisitic regression
and SGDClassifier for each dataset (specifically values of λ used, hard limit on the number of
iterations, etc.). The write up should be self contained in that we should be able to replicate
your results.
• Answer the following questions:
1. Which data representation and algorithm combination yields the best performance (measured in terms of the accuracy, precision, recall and F1 score) and why?
2. Does Multinomial Naive Bayes perform better (again performance is measured in terms
of the accuracy, precision, recall and F1 score) than LR and SGDClassifier on the Bag
of words representation? Explain your yes/no answer.
2
3. Does Discrete Naive Bayes perform better (again performance is measured in terms of
the accuracy, precision, recall and F1 score) than LR and SGDClassifier on the Bernoulli
representation? Explain your yes/no answer.
4. Does your LR implementation outperform the SGDClassifier (again performance is measured in terms of the accuracy, precision, recall and F1 score) or is the difference in
performance minor? Explain your yes/no answer.
References
[1] V. Metsis, I. Androutsopoulos and G. Paliouras, “Spam Filtering with Naive Bayes – Which Naive
Bayes?”. Proceedings of the 3rd Conference on Email and Anti-Spam (CEAS 2006), Mountain
View, CA, USA, 2006.
3
