## Description

Exercise 1.

Flip a biased coin twice. Assume that P[Head] = p. Denote “1” as head, and “0” as tail. Let X be the

maximum of the two numbers, and let Y be the minimum of the two numbers.

(a) Find and sketch the joint PMF pX,Y (x, y).

(b) Find the marginal PMF pX(x) and pY (y).

(c) Find the conditional PMF PX|Y (x | y).

Caution: The setting of this problem is different from Video 7 Problem 1. There we have two persons

throwing a coin 2 times per person (so totally 4 times). Here we throw the coin 2 times only.

c 2020 Stanley Chan. All Rights Reserved. 1

c 2020 Stanley Chan. All Rights Reserved. 2

Exercise 2.

A stock market trader buys 50 shares of stock A and 100 shares of stock B. Let X and Y be the price changes

to A and B, respectively, over certain period of time. Assume that the joint PMF pX,Y (x, y) is uniform over

the set of integers satisfying

−1 ≤ x ≤ 3, −1 ≤ y − x ≤ 1.

(We say that a discrete random variable X is uniformly distributed over a sequence of integers {Na, . . . , Nb}

if pX(x) = 1/(Nb − Na + 1).)

(a) Find the joint PMF pX,Y (x, y), the marginal PMFs pX(x) and pY (y).

(b) Find E[X] and E[Y ].

(c) Determine the trader’s profit.

c 2020 Stanley Chan. All Rights Reserved. 3

Exercise 3.

Let Θ ∼ Uniform[0, 2π].

(a) If X = cos Θ, Y = sin Θ. Are X and Y uncorrelated?

(b) If X = cos(2Θ), Y = sin(2Θ). Are X and Y uncorrelated?

c 2020 Stanley Chan. All Rights Reserved. 4

Exercise 4.

Let

fX,Y (x, y) = (

ce−2x

e

−y

, if 0 ≤ y ≤ x < ∞,

0, otherwise.

(a) Find c.

(b) Find fX(x) and fY (y).

(c) Find E[X] and E[Y ], Var[X] and Var[Y ].

(d) Find E[XY ], Cov(X, Y ) and ρ.

c 2020 Stanley Chan. All Rights Reserved. 5

c 2020 Stanley Chan. All Rights Reserved. 6

Exercise 5.

Let X and Y be two random variables with joint PDF

fX,Y (x, y) = 1

2π

e

− 1

2

(x

2+y

2

)

.

(a) Find the PDF of Z = max(X, Y ).

(b) Find the PDF of Z = min(X, Y ).

You may leave your answers in terms of the Φ(·) function.

c 2020 Stanley Chan. All Rights Reserved. 7

Exercise 6.

Let Y = X + N, where X is the input, N is the noise, and Y is the output of a system. Assume that X and

N are independent random variables. It is given that E[X] = 0, Var[X] = σ

2

X, E[N] = 0, Var[N] = σ

2

N .

(a) Find the correlation coefficient ρ between the input X and the output Y .

(b) Suppose we estimate the input X by a linear function g(Y ) = aY . Find the value of a that minimizes

the mean squared error E[(X − aY )

2

].

(c) Express the resulting mean squared error in terms of η = σ

2

X/σ2

N .

c 2020 Stanley Chan. All Rights Reserved. 8

Exercise 7.

Let X and Y have a joint PDF

fX,Y (x, y) = c(x + y),

for 0 ≤ x ≤ 2 and 0 ≤ y ≤ 2.

(a) Find c, fX(x), fY (y), and E[Y ].

(b) Find fY |X(y|x).

(c) Find P[Y > X | X > 1].

(d) Find E[Y |X = x].

(e) Find E[E[Y |X]], and compare with the E[Y ] computed in (a).

c 2020 Stanley Chan. All Rights Reserved. 9

c 2020 Stanley Chan. All Rights Reserved. 10

Exercise 8.

This is part 1 of a (pretty long) programming exercise.

Image classification is an important problem in computer vision and (probably) the most widely used test

bed problem in artificial intelligence. In this project, we are going to study a simplified version of a common

image classification problem by classifying foreground and background regions in an image. This may sound

challenging as we have not taught these in class. However, the tools you need are actually ready, e.g.,

conditional probability, Bayes’ rule, normal distribution.

The image that we are going to play with can be downloaded from the course website. You tasks for this

exercise are:

(a) Read the image and display the image. Go to the course website, you will see a data file. Download

this .zip file and decompress it into your current working directory. You will be able to see an image

file cat_grass.jpg. To read the image and display the image, you can use imread and im2double in

MATLAB, or plt.imread in Python. Submit the displayed image and your code.

(b) Extract an 8 × 8 patch from the image. To access to the (i, j)th pixel of the image, you can type

Y(i,j). To access the an 8×8 patch at pixel (i, j), you can do Y(i:i+7, j:j+7) for some index i and

j. Extract all the available 8 × 8 patches from the image, and store them as a 64 × K matrix where

K is the number of patches. The following code will be useful.

for i=1:M-8

for j=1:N-8

z = Y(i+[0:7], j+[0:7]);

… % other steps; your job.

end

end

Here, M and N are the number of rows and columns of the image, respectively. No need to worry about

the boundary pixels; just drop them. Hint: Use the command reshape in MATLAB (and Python) to

turn an 8 × 8 patch into a 64 × 1 column vector.

Submit the first 2 columns and the last 2 columns of your data matrix.

(c) Compute the mean vector

µ =

1

K

X

K

k=1

xk, (1)

where xk is the k-th column of your data matrix. Submit your calculated mean vector. Repeat the

calculation using the command mean (MATLAB) or numpy.mean (Python). Submit the mean returned

by this command.

(d) Compute the covariance matrix

Σ =

1

K

X

K

k=1

(xk − µ)(xk − µ)

T

. (2)

Submit your calculated mean vector. Repeat the calculation using the command cov (MATLAB) or

numpy.cov (Python). Submit the covariance returned by this command.

Please attach your results after this page.

c 2020 Stanley Chan. All Rights Reserved. 11