Exam 17: Markov Processes

In Markov analysis,we are concerned with the probability that the

For a situation with weekly dining at either an Italian or Mexican restaurant,

All entries in a matrix of transition probabilities sum to 1.

A state i is a transient state if there exists a state j that is reachable from i,but the state i is not reachable from state j.

​Where is a fundamental matrix,N,used? How is N computed?

If the probability of making a transition from a state is 0,then that state is called a(n)

A state,i,is an absorbing state if,when i = j,p_ij = 1.

​Give two examples of how Markov analysis can aid decision making.

The probability of reaching an absorbing state is given by the

If an absorbing state exists,then the probability that a unit will ultimately move into the absorbing state is given by the steady state probability.

Rent-To-Keep rents household furnishings by the month.At the end of a rental month a customer can: a)rent the item for another month,b)buy the item,or c)return the item.The matrix below describes the month-to-month transition probabilities for 32-inch stereo televisions the shop stocks. What is the probability that a customer who rented a TV this month will eventually buy it?

Henry,a persistent salesman,calls North's Hardware Store once a week hoping to speak with the store's buying agent,Shirley.If Shirley does not accept Henry's call this week,the probability she will do the same next week is .35.On the other hand,if she accepts Henry's call this week,the probability she will not do so next week is .20. a.Construct the transition matrix for this problem. b.How many times per year can Henry expect to talk to Shirley? c.What is the probability Shirley will accept Henry's next two calls if she does not accept his call this week? d.What is the probability of Shirley accepting exactly one of Henry's next two calls if she accepts his call this week?

​What assumptions are necessary for a Markov process to have stationary transition probabilities?

Markov processes use historical probabilities.

A unique matrix of transition probabilities should be developed for each customer.

If a Markov chain has at least one absorbing state,steady-state probabilities cannot be calculated.

Absorbing state probabilities are the same as

All Markov chains have steady-state probabilities.

​Why is a computer necessary for some Markov analyses?

The probability of going from state 1 in period 2 to state 4 in period 3 is