Description
Code:
• For this assignment you may NOT use sklearn .
• You may not use any external libraries except numpy , matplotlib , and
scipy.stats .
2
In this assignment you will experiment with making Bayesian predictions in a simple toy
model. The inputs and outputs are both 1D. The possible predictors are the
functions
x y
fm(x) = { .
0
xm
m = 0
m > 0
This function is plotted below:
5/16/2021 5 Probabilistic Learning
https://www.notion.so/justindomke/5-Probabilistic-Learning-2475c2890fa34710a745e9a25504447c 3/8
We take the following prior over m:
p(m) = .
⎩
⎨
⎧.2
.2
.2
.1
.1
.05
.05
.05
.025
.025
m = 0
m = 1
m = 2
m = 3
m = 4
m = 5
m = 6
m = 7
m = 8
m = 9
The observation model is
6
p(y∣x, m) = N (y∣μ = fm(x), σ ),
2
5/16/2021 5 Probabilistic Learning
https://www.notion.so/justindomke/5-Probabilistic-Learning-2475c2890fa34710a745e9a25504447c 4/8
where is fixed and is a Gaussian distribution with mean and
variance .
σ = .1 N (y∣μ, σ )
2 μ
σ
2
Question 1 (5 points) Implement a function which returns the prior for each value
of . Your function should have the following signature:
p(m) 1
m ∈ {0, 1,⋯ , 9}
def prior(m): # do stuff return p
Give your function directly in your solutions.
Question 2 (5 points) Run the following code, using your prior from the previous
question
for m in range(10): print(“m”, m, “prior(m)”, prior(m))
Give the output directly in your solutions. Also, create a bar-chart of these values, with
m on the x-axis and p(m) on the y-axis. Include that chart here.
Question 3 (10 points) Implement a function that computes the likelihood
for single input/output pair (both scalars). Your function should have the
signature
p(y∣x, m) 4
(x, y)
def likelihood_single(x,y,m): # do stuff return p
Give your function directly in your solutions.
Question 4 (10 points) Implement a function that computes the likelihood of a dataset
5
p(Data∣m) = p(y ∣x , m).
n=1
∏
10
(n) (n)
5/16/2021 5 Probabilistic Learning
https://www.notion.so/justindomke/5-Probabilistic-Learning-2475c2890fa34710a745e9a25504447c 5/8
Your function should have the following signature:
def likelihood(X,Y,m): # do stuff return p
Here X is a 1D array of input values, Y is a 1D array of output values, and m is an
integer. Give your function directly in your solutions.
x.csv 0.2KB
y.csv 0.2KB
Question 5 (5 points) You are given a set of 10 training inputs and outputs. Load these
as X and Y . Then, make a bar chart of the likelihood, with on the x-axis and the
likelihood on the y-axis.
4
m
Question 6 (5 points) Give a mathematical equation for the posterior in
terms of the likelihood and the prior. Make sure your equation is correctly normalized.
p(m∣Data) 2
Question 7 (10 points) Implement a function to compute the posterior given the data.
Your function should have the following signature:
def posterior(X,Y,m): # do stuff return p
Give your function directly in your solutions. 2
Question 8 (5 points) Make a bar chart of your posterior evaluated on each value of .
Make sure your posterior is correctly normalized.
m 2
Question 9 (5 points) Make a function to compute the MAP estimate
mMAP = arg p(m∣Data).
m
max
5/16/2021 5 Probabilistic Learning
https://www.notion.so/justindomke/5-Probabilistic-Learning-2475c2890fa34710a745e9a25504447c 6/8
Your function should have the following signature:
def MAP(X,Y): # do stuff return m
Give your function directly in your solutions.
Question 10 (5 points) What value of do you find, and what is its posterior
probability? (Give specific numbers.)
m 2
Question 11 (5 points) Implement a function to make
predictions using your MAP estimate of . Your function should have the following
signature:
fMAP(x) = fm (x) MAP
m
4
def predict_MAP(x,X,Y): # do stuff return f
Here, X and Y are 1D arrays containing training data, and x is a scalar representing a
point you want to make a prediction for.
x_test.csv 2.4KB
y_test.csv 2.5KB
Question 12 (10 points) You are given a set of 100 test inputs and outputs. Using your
estimate of generated from the training data in x.csv and y.csv , compute
for each element of the test set. Make a plot with the test inputs values in
the x-axis, and the predictions on the y-axis, plotted as circles. Also, plot the
true output values as crosses. Make sure to label all axes and have a legend for the
markers.
mMAP
fMAP(x) x
fMAP(x)
y
Question 13 (5 points) Compute the mean-squared test error of your MAP estimate,
5/16/2021 5 Probabilistic Learning
https://www.notion.so/justindomke/5-Probabilistic-Learning-2475c2890fa34710a745e9a25504447c 7/8
y − f (x ) .
100
1
n=1
∑
100
(
(n) MAP
(n) )
2
Give this error as a number.
Question 14 (10 points) Implement a function to make Bayes predictions
fBayes(x) = p(m∣Data)f (x).
m=0
∑
9
m
Your function should have the following signature:
def predict_Bayes(x,X,Y): # do stuff return f
Give your function directly in your solutions.
Question 15 (10 points) Compute predictions using the Bayes predictor for each
element of the test set (again, using the training data to compute the posterior). Plot
these as circles, comparing against the true outputs plotted with crosses, as in the
question above.
1
Question 16 (5 points) Compute the mean-squared test error of your Bayes estimate,
y − f (x ) .
100
1
n=1
∑
100
(
(n)
Bayes
(n) )
2
Give the final error as a number.
Question 17 (5 points) Is the Bayes or MAP error lower? Given your posterior from the
earlier question, can you reason informally about why this might be true?
5/16/2021 5 Probabilistic Learning
https://www.notion.so/justindomke/5-Probabilistic-Learning-2475c2890fa34710a745e9a25504447c 8/8
