Analysis of best-reply strategies in repeated finite Markov Chains Games

The "best-reply strategy" is a natural and most commonly applied type of actions which players prefer to use during a repeated game. Usually, the behavior of an individual cost-function, when such strategies are applied, turns out to be non-monotonic, and, as the results, to make the conclusion that such strategies lead to some equilibrium point is a non-trivial and delicate task. Moreover, even in repeated games the convergence to a stationary equilibrium is not always guaranteed. Here we show that in the ergodic class of finite controllable Markov Chains Dynamic Games the best reply actions lead obligatory to one of Nash equilibrium points. This conclusion is done by the Lyapunov Games concept which is based on the designing of an individual Lyapunov function (related with an individual cost function) which monotonically decreases (non-increases) during the game. The suggested approach is illustrated by the repeated asynchronous "Prisoner's Dilemma" game with bestreply actions application.