Description
Markov Chains
1. Baron 6.3. (a.i.b.)
In addition, download the R code hw7code.R and use section Problem 1 to answer these questions:
(c) Compute the probability that a dog of the 4th generation (great-great-grand-pup?) is black.
(d) Compute the probability that a dog of the 20th generation is black.
(e) Compute the probability that a dog of the 100th generation is black.
(f) Based on your last answers, what do you think π is?
(g) Based on (f), give P ∞.
You may modify this code to check the rest of your answers.
2. Baron 6.7. For (c) compute π using the method explained on pp. 142-3.
3. Baron 6.1. (a.i.b.)
In addition:
(d) List the absorbing zone in this Markov chain.
4. Modification to Baron 6.1. Use Problem 4 from hw7code.R. Suppose that:
• If there are 0 students in a lab, then the number of students in 1 hour has a 50-50% chance to
increase by 1 or remain unchanged.
• If there is 1 student in a lab, then the number of students in 1 hour has a 50-50% chance to
decrease by 1 or increase by 1.
• If there are 2 students in a lab, then the number of students in 1 hour has a 50-50% chance to
decrease by 1 or remain unchanged.
(a) Write the transition probability matrix for this Markov chain.
(b) Is this a regular Markov chain? Justify your answer.
(c) Suppose there is nobody in the lab at 7am. Give the probabilities that there are 0, 1, or 2 students
working in the lab at 10am.
(d) Suppose there is nobody in the lab at 7am. Give the probabilities that there are 0, 1, or 2 students
working in the lab at 7pm.
(e) Suppose there is nobody in the lab at 7am. Give the probabilities that there are 0, 1, or 2 students
working in the lab at 7am the next morning.
(f) Based on your last answers, give the steady-state probabilities π.
Stat 330-A 1
Homework 7 Markov chains, gamma distribution, Poisson processes due March 14
Gamma probability distribution
5. Confirm in R that for Y ∼ Exponential(λ =
1
2
), i.e. average rate of 1 every 2 time units, that P(Y ≤ 1
time unit) is .3935 by using
> pexp(1, rate=0.5)
Let X be distributed Gamma(α = 5, λ =
1
2
), i.e. time to 5th occurrence with rate of 1 every 2 time
units. Calculate P(X ≤ x) in R using
> pgamma(x, shape=5, rate=0.5).
(a) Compute P(X ≤ 1)
(b) Compute P(X ≤ 2.5) (note: 2.5 is the expected value of X)
(c) Compute P(X ≤ 5)
(d) Compute P(X > 12.5) (probably larger than you thought it was)
6. Baron 4.12.
Poisson process
7. Baron 6.20. (a.i.b.)
8. Baron 6.21.
9. Bill Swerski’s Superfans. Superfan Todd O’Connor (played by the late Chris Farley) has a heart
attack on average once every 4 years. Todd has his first ever physical exam due to new insurance
regulations. His doctor is horrified at his low beef fat-to-pork fat ratio, and his doctor tells him that
he must change his diet. Todd resists, but agrees to change his diet after three heart attacks. What
is the probability that Todd will not have to change his diet for at least 10 years? Assume a Poisson
process is the correct model for Todd’s heart attacks.
Stat 330-A 2
