## Description

Markov Chains

1. Baron 6.3. (a.i.b.)

In addition, download the R code hw7code.R and use section Problem 1 to answer these questions:

(c) Compute the probability that a dog of the 4th generation (great-great-grand-pup?) is black.

(d) Compute the probability that a dog of the 20th generation is black.

(e) Compute the probability that a dog of the 100th generation is black.

(f) Based on your last answers, what do you think π is?

(g) Based on (f), give P ∞.

You may modify this code to check the rest of your answers.

2. Baron 6.7. For (c) compute π using the method explained on pp. 142-3.

3. Baron 6.1. (a.i.b.)

In addition:

(d) List the absorbing zone in this Markov chain.

4. Modification to Baron 6.1. Use Problem 4 from hw7code.R. Suppose that:

• If there are 0 students in a lab, then the number of students in 1 hour has a 50-50% chance to

increase by 1 or remain unchanged.

• If there is 1 student in a lab, then the number of students in 1 hour has a 50-50% chance to

decrease by 1 or increase by 1.

• If there are 2 students in a lab, then the number of students in 1 hour has a 50-50% chance to

decrease by 1 or remain unchanged.

(a) Write the transition probability matrix for this Markov chain.

(b) Is this a regular Markov chain? Justify your answer.

(c) Suppose there is nobody in the lab at 7am. Give the probabilities that there are 0, 1, or 2 students

working in the lab at 10am.

(d) Suppose there is nobody in the lab at 7am. Give the probabilities that there are 0, 1, or 2 students

working in the lab at 7pm.

(e) Suppose there is nobody in the lab at 7am. Give the probabilities that there are 0, 1, or 2 students

working in the lab at 7am the next morning.

(f) Based on your last answers, give the steady-state probabilities π.

Stat 330-A 1

Homework 7 Markov chains, gamma distribution, Poisson processes due March 14

Gamma probability distribution

5. Confirm in R that for Y ∼ Exponential(λ =

1

2

), i.e. average rate of 1 every 2 time units, that P(Y ≤ 1

time unit) is .3935 by using

> pexp(1, rate=0.5)

Let X be distributed Gamma(α = 5, λ =

1

2

), i.e. time to 5th occurrence with rate of 1 every 2 time

units. Calculate P(X ≤ x) in R using

> pgamma(x, shape=5, rate=0.5).

(a) Compute P(X ≤ 1)

(b) Compute P(X ≤ 2.5) (note: 2.5 is the expected value of X)

(c) Compute P(X ≤ 5)

(d) Compute P(X > 12.5) (probably larger than you thought it was)

6. Baron 4.12.

Poisson process

7. Baron 6.20. (a.i.b.)

8. Baron 6.21.

9. Bill Swerski’s Superfans. Superfan Todd O’Connor (played by the late Chris Farley) has a heart

attack on average once every 4 years. Todd has his first ever physical exam due to new insurance

regulations. His doctor is horrified at his low beef fat-to-pork fat ratio, and his doctor tells him that

he must change his diet. Todd resists, but agrees to change his diet after three heart attacks. What

is the probability that Todd will not have to change his diet for at least 10 years? Assume a Poisson

process is the correct model for Todd’s heart attacks.

Stat 330-A 2