Description
Introduction
In this assignment, you’ll get hands-on experience coding and training GANs, GCN (Graph Convolution Network) as well as DQN (Deep Q-learning Network), one of Reinforcement Learning
methods. This assignment is divided into three parts: in the first part, we will implement a specific
type of GAN designed to process images, called a Deep Convolutional GAN (DCGAN). We’ll train
the DCGAN to generate emojis from samples of random noise. In the second part, you will learn
how to implement the vanilla version of GCN and GAT. In the third part, we will implement and
train a DQN agent to learn how to play the CartPole balancing game. It will be fun to see your
model performs much better than you on the simple game :).
Part 1: Deep Convolutional GAN (DCGAN) [4pt]
For the first part of this assignment, we will implement a Deep Convolutional GAN (DCGAN).
A DCGAN is simply a GAN that uses a convolutional neural network as the discriminator, and
a network composed of transposed convolutions as the generator. To implement the DCGAN, we
need to specify three things: 1) the generator, 2) the discriminator, and 3) the training procedure.
We will go over each of these three components in the following subsections.
Open [DCGAN notebook link] on Colab and answer the following questions.
DCGAN
The discriminator in this DCGAN is a convolutional neural network that has the following architecture:
The DCDiscriminator class is implemented for you. We strongly recommend you to carefully
read the code, in particular the __init__ method. The three stages of the generator architectures are implemented using conv and upconv functions respectively, all of which provided in
Helper Modules.
conv1 conv2 conv3 conv4
32
32
3
16
16
32
8
8
64
4
4
128
1
1
1
Discriminator
BatchNorm & ReLU BatchNorm & ReLU BatchNorm & ReLU
Generator
Now, we will implement the generator of the DCGAN, which consists of a sequence of transpose
convolutional layers that progressively upsample the input noise sample to generate a fake image.
The generator has the following architecture:
2
CSC413/2516 Winter 2022 with Prof. Jimmy Ba and Bo Wang Programming Assignment 4
Linear & upconv1 upconv2 upconv3
reshape
32
32
3
16
16
32
8
8
64
4
4
128
BatchNorm & ReLU tanh
Generator
BatchNorm & ReLU BatchNorm & ReLU
100
1
1
1. [1pt] Implementation: Implement this architecture by filling in the __init__ method of
the DCGenerator class, shown below. Note that the forward pass of DCGenerator is already
provided for you.
(Hint: You may find the provided DCDiscriminator useful.)
Note: The original DCGAN generator uses deconv function to expand the spatial dimension.
Odena et al. later found the deconv creates checker board artifacts in the generated samples.
In this assignment, we will use upconv that consists of an upsampling layer followed by conv2D
to replace the deconv module (analogous to the conv function used for the discriminator
above) in your generator implementation.
Training Loop
Next, you will implement the training loop for the DCGAN. A DCGAN is simply a GAN with a
specific type of generator and discriminator; thus, we train it in exactly the same way as a standard
GAN. The pseudo-code for the training procedure is shown below. The actual implementation is
simpler than it may seem from the pseudo-code: this will give you practice in translating math to
code.
Algorithm 1 Regular GAN Training Loop Pseudocode
1: procedure TrainGAN
2: Draw m training examples {x
(1), . . . , x(m)} from the data distribution pdata
3: Draw m noise samples {z
(1), . . . , z(m)} from the noise distribution pz
4: Generate fake images from the noise: G(z
(i)
) for i ∈ {1, . . . .m}
5: Compute the discriminator loss (negative log likelihood):
L
(D) = −
1
m
Xm
i=1
h
log D
x
(i)
+ log
1 − D
G(z
(i)
)
i
6: Update the parameters of the discriminator
7: Draw m new noise samples {z
(1), . . . , z(m)} from the noise distribution pz
8: Generate fake images from the noise: G(z
(i)
) for i ∈ {1, . . . .m}
9: Compute the generator loss (negative log likelihood):
L
(G) = −
1
m
Xm
i=1
log
D(G(z
(i)
))
10: Update the parameters of the generator
3
CSC413/2516 Winter 2022 with Prof. Jimmy Ba and Bo Wang Programming Assignment 4
1. [1pt] Implementation: Fill in the gan_training_loop_regular function in the GAN section of the notebook.
There are 5 numbered bullets in the code to fill in for the discriminator and 3 bullets for
the generator. Note that in the original GAN paper, we want to train the Generator by
minimizing log
1 − D(G(z
(i)
))
in an effort to generate better fakes. However, this was shown
by Goodfellow to not provide sufficient gradients, especially early in the learning process. As
a fix, we instead wish to maximize log
D(G(z
(i)
))
, which is also refered as the non-saturating
GAN Loss.
Experiment
1. [1pt] We will train a DCGAN to generate Windows (or Apple) emojis in the Training – GAN
section of the notebook. By default, the script runs for 20000 iterations, and should take
approximately 20 minutes on Colab. The script saves the output of the generator for a fixed
noise sample every 200 iterations throughout training; this allows you to see how the generator
improves over time. How does the generator performance evolve over time? Include in
your write-up some representative samples (e.g. one early in the training, one
with satisfactory image quality, and one towards the end of training, and give
the iteration number for those samples. Briefly comment on the quality of the
samples.
2. [1pt] Multiple techniques can be used to stabilize GAN training. Least-squares GAN is one
of the many techniques [Mao et al., 2017]. Fill in the gan_training_loop_leastsquares
function in the GAN section of the notebook. Compared to the regular GAN training loop
you have implemented, you will only need to modify the discriminator loss and the generator
loss to the following:
L
(D)
ls =
1
2m
Xm
i=1
D(x
(i)
) − 1
2
+
1
2m
Xm
i=1
D(G(z
(i)
))2
(1)
L
(G)
ls =
1
m
Xm
i=1
D(G(z
(i)
)) − 1
2
(2)
After the implementation, try turn on the least_squares_gan flag in the args_dict and
train the model again. Are you able to stabilize the training? Briefly explain in 1∼2 sentences
why the least squares GAN can help. You are welcome to check out the related blog posts
for LSGAN.
