Description
Questions:
1. Let X1, X2, …, Xn be n > 0 independent identically distributed random variables with cdf FX(x) and
pdf fX(x) = F
0
X(x). Derive an expression for the cdf and pdf of Y1 = max(X1, X2, …, Xn) and Y2 =
min(X1, X2, …, Xn) in terms of FX(x). [10 points]
2. We say that a random variable X belongs to a Gaussian mixture model (GMM) if X ∼
PK
i=1 piN (µi
, σ2
i
)
where pi
is the ‘mixing probability’ for each of the K constituent Gaussians, with PK
i=1 pi = 1; ∀i, 0 ≤ pi ≤ 1.
To draw a sample from a GMM, we do the following: (1) One of the K Gaussians is randomly chosen as per
the PMF {p1, p2, …, pK} (thus, a Gaussian with a higher mixing probability has a higher chance of being
picked). (2) Let the index of the chosen Gaussian be (say) m. Then, you draw the value from N (µm, σ2
m).
If X belongs to a GMM as defined here, obtain expressions for E(X), Var(X) and the MGF of X.
Now consider a random variable of the form Xi ∼ N (µi
, σ2
i
) for each i ∈ {1, 2, …, K}. Define another random
variable Z =
PK
i=1 piXi where {Xi}
K
i=1 are independent random variables. Derive an expression for
E(Z), Var(Z) and the PDF, MGF of Z. [2+2+2+2+2+2+3=15 points]
3. Using Markov’s inequality, prove the following one-sided version of Chebyshev’s inequality for random variable X with mean µ and variance σ
2
: P(X − µ ≥ τ ) ≤
σ
2
σ
2 + τ
2
if τ > 0, and P(X − µ ≥ τ ) ≥ 1 −
σ
2
σ
2 + τ
2
if τ < 0. [15 points]
4. Given stuff you’ve learned in class, prove the following bounds: P(X ≥ x) ≤ e
−txφX(t) for t > 0, and
P(X ≤ x) ≤ e
−txφX(t) for t < 0. Here φX(t) represents the MGF of random variable X for parameter t.
1
Now consider that X denotes the sum of n independent Bernoulli random variables X1, X2, …, Xn where
E(Xi) = pi
. Let µ =
Pn
i=1 pi
. Then show that P(X > (1 + δ)µ) ≤
e
µ(e
t−1)
e
(1+δ)tµ for any t ≥ 0, δ > 0. You may
use the inequality 1 + x ≤ e
x
. Further show how to tighten this bound by choosing an optimal value of t.
[15 points]
5. Consider N independent random variables X1, X2, …, XN , such that each variable Xi takes on the values
1, 2, 3, 4, 5 with probability 0.05, 0.4, 0.15, 0.3, 0.1 respectively. For different values of
N ∈ {5, 10, 20, 50, 100, 200, 500, 1000, 5000, 10000}, do as follows:
(a) Plot the (empirically determined) distribution of the average of these random variables (X
(N)
P
avg =
N
i=1 Xi/N) in the form of a histogram with 50 bins.
(b) Empirically determine the CDF of X
(N)
avg using the ecdf command of MATLAB (this is called the
empirical CDF). On a separate figure, plot the empirical CDF. On this, overlay the CDF of a Gaussian
having the same mean and variance as X
(N)
avg . To get the CDF of the Gaussian, use the normcdf function
of MATLAB.
(c) Let E(N) denote the empirical CDF and Φ(N) denote the Gaussian CDF. Compute the maximum
absolute difference (MAD) between E(N)
(x) and Φ(N)
(x) numerically, at all values x returned by ecdf.
For this, read the documentation of ecdf carefully. Plot a graph of MAD as a function of N. [3+3+4
= 10 points]
6. Read in the images T1.jpg and T2.jpg from the homework folder using the MATLAB function imread and
cast them as a double array. These are magnetic resonance images of a portion of the human brain, acquired
with different settings of the MRI machine. They both represent the same anatomical structures and are
perfectly aligned (i.e. any pixel at location (x, y) in both images represents the exact same physical entity).
Consider random variables I1, I2 which denote the pixel intensities from the two images respectively. Write
a piece of MATLAB code to shift the second image along the X direction by tx pixels where tx is an integer
ranging from -10 to +10. While doing so, assign a value of 0 to unoccupied pixels. For each shift, compute
the following measures of dependence between the first image and the shifted version of the second image:
the correlation coefficient ρ,
a measure of dependence called quadratic mutual information (QMI) defined as P
i1
P
i2
(pI1I2
(i1, i2) −
pI1
(i1)pI2
(i2))2
, where pI1I2
(i1, i2) represents the normalized joint histogram (i.e., joint pmf) of I1 and
I2 (‘normalized’ means that the entries sum up to one).
For computing the joint histogram, use a bin-width of 10 in both I1 and I2. For computing the marginal
histogram, you need to integrate the joint histogram along one of the two directions respectively. You should
write your own joint histogram routine in MATLAB – do not use any inbuilt functions for it. Plot a graph
of the values of ρ versus tx, and another graph of the values of QMI versus tx.
Repeat exactly the same steps when the second image is a negative of the first image, i.e. I2 = 255 − I1.
Comment on all the plots. In particular, what do you observe regarding the relationship between the
dependence measures and the alignment between the two images? Your report should contain all four plots
labelled properly, and the comments on them as mentioned before. [25 points]
7. Derive the covariance matrix of a multinomial distribution using moment generating functions. You are not
allowed to use any other method. Since a covariance matrix C is square and symmetric, it is enough to
derive expression for the diagonal elements Cii and the off-diagonal elements Cij , i 6= j. [10 points]
2
