Markov Chains

Objectives

-Objective1: Recognize and prove that a stochastic process is a Markov chain.
-objective2: Analyze the static structure of a Markov chain (i.e. establish the associated transition graph, identify the communication class structure, show the recurrence or transience of chain states, calculate class periodicities).
-Objective3: Describe the limit behavior of an ergodic chain (by calculating the stationary law, possibly reversible; quote and apply the limit theorems of the course).

Course outline

Definition of a discrete-time homogeneous Markov chain on a discrete state space. Illustration with fundamental models introduced in various modeling contexts. Chapman-Kolmogorov equation and conditioning formulas. State classification, periodicity, attainment time, recurrence and transience. Stationary law and limit theorems, reversibility of a Markov chain.

Prerequisites

Basic probability course