- Home
- COMP 3670/6670
- Assignment 4 COMP3670: Introduction to Machine Learning

~~$40.00~~ $24.00

Category: COMP 3670/6670

Description

5/5 - (4 votes)

Question 1 Continuous Bayesian Inference 5+5+2+4+4+6+6+5=37 credits

Let X be a random variable representing the outcome of a biased coin with possible outcomes X =

{0, 1}, x ∈ X . The bias of the coin is itself controlled by a random variable Θ, with outcomes2

θ ∈ θ,

where

θ = {θ ∈ R : 0 ≤ θ ≤ 1}

The two random variables are related by the following conditional probability distribution function of

X given Θ.

p(X = 1 | Θ = θ) = θ

p(X = 0 | Θ = θ) = 1 − θ

We can use p(X = 1 | θ) as a shorthand for p(X = 1 | Θ = θ).

We wish to learn what θ is, based on experiments by flipping the coin.

We flip the coin a number of times.3 After each coin flip, we update the probability distribution for θ

to reflect our new belief of the distribution on θ, based on evidence.

Suppose we flip the coin n times, and obtain the sequence of coin flips 4 x1:n.

a) Compute the new PDF for θ after having observed n consecutive ones (that is, x1:n is a sequence

where ∀i.xi = 1), for an arbitrary prior pdf p(θ). Simplify your answer as much as possible.

b) Compute the new PDF for θ after having observed n consecutive zeros, (that is, x1:n is a sequence

where ∀i.xi = 0) for an arbitrary prior pdf p(θ). Simplify your answer as much as possible.

c) Compute p(θ|x1:n = 1n

) for the uniform prior p(θ) = 1.

d) Compute the expected value µn of θ after observing n consecutive ones, with a uniform prior

p(θ) = 1. Provide intuition explaining the behaviour of µn as n → ∞.

1For example, asserting that R 1

0

x

2

x

3 + 2x

dx = 2/3 with no working out is adequate, as you could just plug the

integral into Wolfram Alpha using the command Integrate[x^2(x^3 + 2x),{x,0,1}]

2For example, a value of θ = 1 represents a coin with 1 on both sides. A value of θ = 0 represnts a coin with 0 on

both sides, and θ = 1/2 represents a fair, unbaised coin.

3The coin flips are independent and identically distributed (i.i.d).

4We write x1:n as shorthand for the sequence x1x2 . . . xn.

WhatsApp us