TY - JOUR
T1 - First passage problems for nonstationary discrete-time stochastic control systems
AU - Guo, Xianping
AU - Hernández-del-Valle, Adrián
AU - Hernández-Lerma, Onésimo
N1 - Funding Information:
The research of X. Guo was supported by NSFC, GDUPS, and GPK-LCS and O. Hernández-Lerma by CONACyT grant 104001.
PY - 2012
Y1 - 2012
N2 - This paper concerns first passage problems for nonstationary nonlinear discrete-time stochastic control systems. The state and control spaces are general Borel spaces, the transition probabilities are nonstationary, the costs/rewards are time-varying and may be unbounded. The optimal control problem is to minimize the expected discounted costs incurred until the first passage time to some target set, in which the discount factors are allowed to be both time- and state-dependent. Under reasonably mild conditions we establish the so-called first passage optimality equations, and prove that the optimal cost/reward functions satisfy the optimality equations. Furthermore, from the optimality equations we show the existence of optimal Markov policies.
AB - This paper concerns first passage problems for nonstationary nonlinear discrete-time stochastic control systems. The state and control spaces are general Borel spaces, the transition probabilities are nonstationary, the costs/rewards are time-varying and may be unbounded. The optimal control problem is to minimize the expected discounted costs incurred until the first passage time to some target set, in which the discount factors are allowed to be both time- and state-dependent. Under reasonably mild conditions we establish the so-called first passage optimality equations, and prove that the optimal cost/reward functions satisfy the optimality equations. Furthermore, from the optimality equations we show the existence of optimal Markov policies.
KW - Discrete-time systems
KW - First passage time
KW - Nonlinear stochastic systems
KW - Nonstationary stochastic systems
KW - Target set
UR - http://www.scopus.com/inward/record.url?scp=84874622974&partnerID=8YFLogxK
U2 - 10.3166/EJC.18.528-538
DO - 10.3166/EJC.18.528-538
M3 - Artículo
SN - 0947-3580
VL - 18
SP - 528
EP - 538
JO - European Journal of Control
JF - European Journal of Control
IS - 6
ER -