Description
1. [Marginalization] Consider the following distribution p(x, y, z) of three discrete random
variables X, Y and Z.
(X, Y)
Z
1 2 3
(1, 1) 0.2 0.2 0.1
(1, 2) 0.02 0.1 0.05
(2, 1) 0.03 0.05 0.03
(2, 2) 0.1 0.07 0.05
(a) [2pt] Compute the marginal distributions p(x, y) and p(z).
(b) [2pt] Compute the expectation E[X + Y
2
] and E[Z].
(c) [2pt] The conditional distributions p(x|Y = 1) and p(z | X = 1).
(d) [2pt] The conditional expectations E[X | Y = 1] and E[Z | X = 1].
2
2. [Expectation] Consider two random variables X and Y of joint distribution p(x, y)
and finite supports X and Y.
(a) [2pt] Prove that the expected value of the conditional expected value of X given
Y is the same as the expected value of X, i.e.,
EX[X] = EY [EX[X|Y ]] .
(b) [2pt] Prove that the covariance of X and Y can be computed as follows:
cov(X, Y ) = E[XY ] − E[X] E[Y ] .
(c) [2pt] Prove or disprove that if cov(X, Y ) = 0, then X and Y are independent.
3
3. [Mixture of Gaussian] Let X be a Bernoulli random variable of mean α ∈ [0, 1].
Consider a random vector Y =
Y1
Y2
∈ R
2
such that p(y|X = 0) = N (y; µ, Σ) and
p(y|X = 1) = N (y; µ
0
, Σ
0
) where µ =
µ1
µ2
∈ R
2
, µ
0 =
µ
0
1
µ
0
2
∈ R
2
, Σ =
Σ1,1 Σ1,2
Σ2,1 Σ2,2
∈
R
4 and Σ0 =
Σ
0
1,1 Σ
0
1,2
Σ
0
2,1 Σ
0
2,2
∈ R
4
.
(a) [1pt] Write the distribution of Y using N (y; µ, Σ) and N (y; µ
0
, Σ
0
).
(b) [2pt] Find the set of α making X and Y independent to each other regardless of
µ, µ0
, Σ and Σ0
.
(c) [2pt] Compute the marginal distributions of Y1 and Y2.
(d) [2pt] Compute the means of Y1 and Y2.
(e) [2pt] Compute the variances of Y1 and Y2.
4
4. [2pt; Bayes’ theorem] There are two bags. The first bag contains four mangos and
two apples; the second bag contains four mangos and four apples. We also have a
biased coin, which shows heads with probability 0.6 and tail with probability 0.4. If
the coin shows heads. we pick a fruit at random from bag 1; otherwise we pick a fruit
at random from bag 2. Your friend flips the coin (you cannot see the result), picks a
fruit at random from the corresponding bag, and presents you a mango. What is the
probability that the mango was picked from bag 2? (Show your calculations as well as
giving the final result.)
5
5. [Basic understanding of convexity] Consider whether the following statements are true
or false:
(a) [1pt] The intersection of any two convex sets is convex.
(b) [1pt] The union of any two convex sets is convex.
(c) [1pt] The difference of a convex set A from another convex set B is convex
(d) [1pt] The sum of any two convex functions is convex.
(e) [1pt] The difference of any two convex functions is convex.
(f) [1pt] The product of any two convex functions is convex.
(g) [1pt] The maximum of any two convex functions is convex
6
6. [Basic understanding of calculus] Consider f(x) = x
3 − 2x
2 + x.
(a) [2pt] Find all the stationary points of f(x), i.e., values of x making ∂f(x)/∂x = 0.
(b) [2pt] Find the largest interval of x on which f(x) is convex.
(c) [1pt] Minimize f(x) on [−1, 3]. (answer the minimal objective and the corresponding solution.)
(d) [1pt] Minimize f(x) on [1/2, 3]. (answer the minimal objective and the corresponding solution.)
7