Description
1 Transfer Learning and TensorFlow API
This assignment will give you a chance to familiarize with transfer learning and with the different
techniques for monitoring and controlling the training process in tensorflow.
You must use Keras with the tensorflow backend, i.e., the package tensorflow.keras. For
this assignment, you may use other tensorflow packages and scikit-learn and scikit-image but not
other deep learning frameworks, e.g., caffe, pytorch, theano etc.
2 VGG-16 [3]
This part of the assignment is a continuation from Assignment 2 and as such you will continue
to work with the Extended Outex texture dataset from the University of Oulu and LAGIS-FRE
CNRS. For download instruction and training/test split, please see Assignment 2 if you have not
yet already downloaded and installed the images.
For this part adapt the the VGG-16 network for the task. The pre-trained network is available
from tf.keras.applications.vgg16. You want to suitably remove some layers (importing with
include top=False is a good start but less layers are likely sufficient and will run much faster)
and add fully-connected layer(s) and a softmax classifier at the output. You will need to train the
new layers with the weights of the existing VGG layers fixed. Once you have a working classifier
for the task, try to improve the classification result by training some layers a bit more (typically
the higher-level) layers.
3 MobileNetV2 [5]
This part of the assignment will use parts of the Caltech-UCSD Birds-200-2011 (CUB-200-2011)
dataset. This datset contains images of 200 different species of birds and different annotations
per image. We will use the bird species (categories) and the bounding box annotation. This will
give a chance to consider a regression task. As the dataset is of medium size (1.1 GB), we will be
working with a subset, in particular, we will only work with the following bird species: cardinal,
american goldfinch, evening grosbeak, song sparrow, tree sparrow, house sparrow, fox sparrow,
canada warbler, myrtle warbler, house wren, red-winged blackbird, boat tailed grackle.
Similar to Section 2, adapt a pre-trained network for the task. Here, we will use MobileNetV2.
You will have to create and compare different versions of the network.
1
3.1 Regularization
Design your model for classification with three different regularizations using dropout, batch normalization and l1-regularization. Briefly compare the obtained models in terms of accuracy. Note
that the regularization only applies to layers where weights have not been frozen.
3.2 Optimizers
Pick the best performing model from Section 3.1 and evaluate at least three different optimizers. One optimizer must be stochastic gradient descent (SGD). Briefly compare the optimizer
performance.
3.3 Multi-task Learning
Revise your model from Section 3.2 to add a regression task to the output. You will have to decide
on which layers to share between the two tasks. Discuss your observations.
3.4 Bonus: Additional Annotations
For bonus extend your model from Section 3.3 to predict further labels: either part labels or
attribute labels. Describe your approach and evaluate how successful your attempts were.
4 Submission
You will need to submit your solution in a Jupyter file, do not submit the data. Make sure you
have run all the cells. All text must be embedded in the Jupyter file, I will not look at separately
submitted text files. If your Jupyter file needs a local python file to run, please submit it as
well.Assignment submission is only though Virtual Campus by the deadline. No late submissions
are allowed, you can submit multiple times but only your last submission is kept and marked.
2
