사용자:Sulgi Kim/연습장

예

 이 부분의 본문은 마르코프 체인의 예입니다.

간단한 예의 상태 다이아그램 for a simple example is shown in the figure on the right, using a directed graph to picture the state transitions. 오른쪽 그림은 간단한 예를 상태전이를 나타내는 방향그래프를 이용하여 상태다이아그램(state diagram)으로 나타낸 것이다. The states represent whether a hypothetical stock market is exhibiting a bull market, bear market, or stagnant market trend during a given week. 각 상태는 가상의 주식 시장이 한 주간 상승세인지 하락세인지 혼조세인지를 나타낸다. 그림을 보면, 강세 주간 이후 다음 주간은, 90% 가 강세이고, 7.5% 는 약세 이며, 2.5%는 혼조세이다. 상태공간을 {1 = 상승세, 2 = 하락세, 3 = 혼조세} 로 나타내면, 예시의 전이행렬은

$P={\begin{bmatrix}0.9&0.075&0.025\\0.15&0.8&0.05\\0.25&0.25&0.5\end{bmatrix}}.$

The distribution over states can be written as a stochastic row vector x with the relation x^(n + 1) = x⁽ⁿ⁾P. So if at time n the system is in state 2 (bear), then three time periods later, at time n + 3 the distribution is

${\begin{aligned}x^{(n+3)}&=x^{(n+2)}P=\left(x^{(n+1)}P\right)P\\\\&=x^{(n+1)}P^{2}=\left(x^{(n)}P^{2}\right)P\\&=x^{(n)}P^{3}\\&={\begin{bmatrix}0&1&0\end{bmatrix}}{\begin{bmatrix}0.9&0.075&0.025\\0.15&0.8&0.05\\0.25&0.25&0.5\end{bmatrix}}^{3}\\&={\begin{bmatrix}0&1&0\end{bmatrix}}{\begin{bmatrix}0.7745&0.17875&0.04675\\0.3575&0.56825&0.07425\\0.4675&0.37125&0.16125\\\end{bmatrix}}\\&={\begin{bmatrix}0.3575&0.56825&0.07425\end{bmatrix}}.\end{aligned}}$

Using the transition matrix it is possible to calculate, for example, the long-term fraction of weeks during which the market is stagnant, or the average number of weeks it will take to go from a stagnant to a bull market. Using the transition probabilities, the steady-state probabilities indicate that 62.5% of weeks will be in a bull market, 31.25% of weeks will be in a bear market and 6.25% of weeks will be stagnant, since:

$\lim _{N\to \infty }\,P^{N}={\begin{bmatrix}0.625&0.3125&0.0625\\0.625&0.3125&0.0625\\0.625&0.3125&0.0625\\\end{bmatrix}}$

A thorough development and many examples can be found in the on-line monograph Meyn & Tweedie 2005.

The appendix of Meyn 2007, also available on-line, contains an abridged Meyn & Tweedie.

A finite state machine can be used as a representation of a Markov chain. Assuming a sequence of independent and identically distributed input signals (for example, symbols from a binary alphabet chosen by coin tosses), if the machine is in state y at time n, then the probability that it moves to state x at time n + 1 depends only on the current state.