Description
Exercise 1.
Flip a biased coin twice. Assume that P[Head] = p. Denote “1” as head, and “0” as tail. Let X be the
maximum of the two numbers, and let Y be the minimum of the two numbers.
(a) Find and sketch the joint PMF pX,Y (x, y).
(b) Find the marginal PMF pX(x) and pY (y).
(c) Find the conditional PMF PX|Y (x | y).
Caution: The setting of this problem is different from Video 7 Problem 1. There we have two persons
throwing a coin 2 times per person (so totally 4 times). Here we throw the coin 2 times only.
c 2020 Stanley Chan. All Rights Reserved. 1
c 2020 Stanley Chan. All Rights Reserved. 2
Exercise 2.
A stock market trader buys 50 shares of stock A and 100 shares of stock B. Let X and Y be the price changes
to A and B, respectively, over certain period of time. Assume that the joint PMF pX,Y (x, y) is uniform over
the set of integers satisfying
−1 ≤ x ≤ 3, −1 ≤ y − x ≤ 1.
(We say that a discrete random variable X is uniformly distributed over a sequence of integers {Na, . . . , Nb}
if pX(x) = 1/(Nb − Na + 1).)
(a) Find the joint PMF pX,Y (x, y), the marginal PMFs pX(x) and pY (y).
(b) Find E[X] and E[Y ].
(c) Determine the trader’s profit.
c 2020 Stanley Chan. All Rights Reserved. 3
Exercise 3.
Let Θ ∼ Uniform[0, 2π].
(a) If X = cos Θ, Y = sin Θ. Are X and Y uncorrelated?
(b) If X = cos(2Θ), Y = sin(2Θ). Are X and Y uncorrelated?
c 2020 Stanley Chan. All Rights Reserved. 4
Exercise 4.
Let
fX,Y (x, y) = (
ce−2x
e
−y
, if 0 ≤ y ≤ x < ∞,
0, otherwise.
(a) Find c.
(b) Find fX(x) and fY (y).
(c) Find E[X] and E[Y ], Var[X] and Var[Y ].
(d) Find E[XY ], Cov(X, Y ) and ρ.
c 2020 Stanley Chan. All Rights Reserved. 5
c 2020 Stanley Chan. All Rights Reserved. 6
Exercise 5.
Let X and Y be two random variables with joint PDF
fX,Y (x, y) = 1
2π
e
− 1
2
(x
2+y
2
)
.
(a) Find the PDF of Z = max(X, Y ).
(b) Find the PDF of Z = min(X, Y ).
You may leave your answers in terms of the Φ(·) function.
c 2020 Stanley Chan. All Rights Reserved. 7
Exercise 6.
Let Y = X + N, where X is the input, N is the noise, and Y is the output of a system. Assume that X and
N are independent random variables. It is given that E[X] = 0, Var[X] = σ
2
X, E[N] = 0, Var[N] = σ
2
N .
(a) Find the correlation coefficient ρ between the input X and the output Y .
(b) Suppose we estimate the input X by a linear function g(Y ) = aY . Find the value of a that minimizes
the mean squared error E[(X − aY )
2
].
(c) Express the resulting mean squared error in terms of η = σ
2
X/σ2
N .
c 2020 Stanley Chan. All Rights Reserved. 8
Exercise 7.
Let X and Y have a joint PDF
fX,Y (x, y) = c(x + y),
for 0 ≤ x ≤ 2 and 0 ≤ y ≤ 2.
(a) Find c, fX(x), fY (y), and E[Y ].
(b) Find fY |X(y|x).
(c) Find P[Y > X | X > 1].
(d) Find E[Y |X = x].
(e) Find E[E[Y |X]], and compare with the E[Y ] computed in (a).
c 2020 Stanley Chan. All Rights Reserved. 9
c 2020 Stanley Chan. All Rights Reserved. 10
Exercise 8.
This is part 1 of a (pretty long) programming exercise.
Image classification is an important problem in computer vision and (probably) the most widely used test
bed problem in artificial intelligence. In this project, we are going to study a simplified version of a common
image classification problem by classifying foreground and background regions in an image. This may sound
challenging as we have not taught these in class. However, the tools you need are actually ready, e.g.,
conditional probability, Bayes’ rule, normal distribution.
The image that we are going to play with can be downloaded from the course website. You tasks for this
exercise are:
(a) Read the image and display the image. Go to the course website, you will see a data file. Download
this .zip file and decompress it into your current working directory. You will be able to see an image
file cat_grass.jpg. To read the image and display the image, you can use imread and im2double in
MATLAB, or plt.imread in Python. Submit the displayed image and your code.
(b) Extract an 8 × 8 patch from the image. To access to the (i, j)th pixel of the image, you can type
Y(i,j). To access the an 8×8 patch at pixel (i, j), you can do Y(i:i+7, j:j+7) for some index i and
j. Extract all the available 8 × 8 patches from the image, and store them as a 64 × K matrix where
K is the number of patches. The following code will be useful.
for i=1:M-8
for j=1:N-8
z = Y(i+[0:7], j+[0:7]);
… % other steps; your job.
end
end
Here, M and N are the number of rows and columns of the image, respectively. No need to worry about
the boundary pixels; just drop them. Hint: Use the command reshape in MATLAB (and Python) to
turn an 8 × 8 patch into a 64 × 1 column vector.
Submit the first 2 columns and the last 2 columns of your data matrix.
(c) Compute the mean vector
µ =
1
K
X
K
k=1
xk, (1)
where xk is the k-th column of your data matrix. Submit your calculated mean vector. Repeat the
calculation using the command mean (MATLAB) or numpy.mean (Python). Submit the mean returned
by this command.
(d) Compute the covariance matrix
Σ =
1
K
X
K
k=1
(xk − µ)(xk − µ)
T
. (2)
Submit your calculated mean vector. Repeat the calculation using the command cov (MATLAB) or
numpy.cov (Python). Submit the covariance returned by this command.
Please attach your results after this page.
c 2020 Stanley Chan. All Rights Reserved. 11
