CMPT 728 Deep Learning Assignment 1

$30.00

Category: You will Instantly receive a download link for .zip solution file upon Payment || To Order Original Work Click Custom Order?

Description

5/5 - (1 vote)

A. Conceptual Exercises (50 points)
A.1 Gradients and Backpropagation (5 +5 = 10 points)
Given a training example (x,y), and a set of linear weights w, find the gradient of the
loss function with respect to w for the following loss functions (definition in the
slides).

1. Least-squares error (see also Exercise 1.6 in the text)
2. Cross-entropy
Describe informally but clearly how you can incorporate these gradients in the
backpropagation algorithm described in class (see lecture slides).

A.2 Broadcasting (10 points)
Compute the following:
!
0.5 1.5
3.5 2 ) ! 0.5 1
3 2 ) + ( 1.25 3 )
You should assume broadcasting so the computation is well defined.

A.3 Trace Backpropagation (30 points)
Consider a neural net with one hidden layer, two inputs a and b, one hidden unit c,
and one output unit d. The activation function is the sigmoid for each node. This
network has five weights (wac, wbc, w0c, wcd, w0d), where w0x represents the bias or
threshold weight for unit x.

Initialize these weights to the values (.2, .1, .2, .1,.2), then
give their values after each of the first two training iterations of Backpropagation
algorithm.

Assume learning rate (step size) of 0.1, stochastic (incremental) gradient
descent (without momentum), least-squares for the loss/error function. The input
are the following training examples:
Data Point a b d
x1 1 0 1
x2 0 1 0

1. Using the notation in the slides, show the formulas for computing the
following quantities, for each node x:
• ax
• D[x]
• Weight update forwxy

2. Fill in the following table using the formulas from the slides. You can expand
this to include more information (e.g. derivatives of activation functions) if
you like.

Data Point ac D[c] ad D[d] w0c wac wbc wcd w0d
x1
x2

B. Project Part (100 points):
Please see the assignment1_proj.ipynb and the assignment1_proj.pdf
files.