Description
1
1. [2 points] Adapted from Exercise 3.1 of FCMA p.133:
For α, β = 1, the beta distribution becomes uniform between 0 and 1. In particular, if the probability
of a coin landing heads is given by r and a beta prior is placed over r, with parameters α = 1, β = 1,
this prior can be written as follows:
p(r) = 1 (0 ≤ r ≤ 1)
Using this prior, compute the posterior density for r if y heads are observed in N tosses (i.e., multiply
this prior by the binomial likelihood and manipulate the result to obtain something that looks like a
beta density).
Solution.
2. [2 points] Adapted from Exercise 3.2 of FCMA p.134:
Repeat the previous exercise for the following prior, also a particular form of the Beta density:
p(r) = (
2r 0 ≤ r ≤ 1
0 otherwise
What are the values of the prior parameters α and β that result in p(r) = 2r?
Solution.
3. [3 points] Adapted from Exercise 3.5 of FCMA p.134:
If a random variable R has a beta density
p(r) = Γ(α + β)
Γ(α)Γ(β)
r
α−1
(1 − r)
β−1
,
derive an expression for the expected value of r, Ep(r){r}. You will need the following identity for the
gamma function:
Γ(n + 1) = nΓ(n).
Hint: use the fact that
Z r=1
r=0
r
α−1
(1 − r)
b−1 dr =
Γ(a)Γ(b)
Γ(a + b)
.
Solution.
4. [5 points; Required only for Graduates] Adapted from Exercise 3.12 of FCMA p.135:
When performing a Bayesian analysis of the Olympics data, we assumed that σ
2 was known. If instead
we assume that w is known and an inverse Gamma prior is placed on σ
2
p(σ
2
|α, β) = β
α
Γ(α)
(σ
2
)
−α−1
exp
−
β
σ
2
,
the posterior over σ
2 will also be inverse Gamma. Derive the posterior parameters.
Solution.
2
5. [5 points] Adapted from Exercise 4.2 of FCMA p.165-166:
In Chapter 3, we computed the posterior density over r, the probability of a coin giving heads, using a
beta prior and a binomial likelihood. Recalling that the beta prior, with parameters α and β, is given
by
p(r|α, β) = Γ(α + β)
Γ(α)Γ(β)
r
α−1
(1 − r)
β−1
and the binomial likelihood, assuming y heads in N throws, is given by
p(y|r, N) =
y
N
r
y
(1 − r)
N−y
compute the Laplace approximation to the posterior. (Note, you should be able to obtain a closed-form
solution for the MAP value, ˆr, by setting the log posterior to zero, differentiating, equating to zero
and solving for r.)
Solution.
6. [4 points] Adapted from Exercise 4.3 of FCMA p.166:
In the previous exercise you computed the Laplace approximation to the true beta posterior. In this
problem, plot both the true beta posterior and the Laplace approximation for the three sets of values:
1. α = 5, β = 5, N = 20, and y = 10,
2. α = 3, β = 15, N = 10, and y = 3,
3. α = 1, β = 30, N = 10, and y = 3.
Be sure to clearly indicate the values in your plot captions. Include how the two distributions (the true
beta posterior and the Laplace approximation) compare in each case. Include the python script you use
to generate these plots; the script should be named plot laplace approx.py. Suggestion: for plotting the beta and Gaussian (Normal) distributions, use scipy.stats.beta and scipy.stats.normal
to create the beta and Gaussian random variables, and use the pdf(x) method for each to generate the
curves. Note that for scipy.stats.normal, the mean is the location (loc) parameter, and the sigma
is the scale parameter. Also, scipy.stats.normal expects the scale parameter to be the standard
deviation (i.e., take the square root: math.sqrt(x)) of the variance you’ll compute for the Laplace
approximation.
Solution.
7. [4 points] Adapted from Exercise 4.4 of FCMA p.166:
Given the expression for the area of a circle, A = πr2
, and using only uniformly distributed random
variates, devise a sampling approach for computing π. Describe your method in detail and provide you
script to do the estimation – this script should be called pi sample estimate.py.
Solution.
3
