Optimization problems in chemical reactions using continuous-time Markov chains

In this paper we generalize two models for chemical reactions, based on continuous-time Markov chains, into continuous-time Markov decision process. We propose a mathematical optimization approach for solving an average optimality criterion in state-discrete continuous-time Markov decision process. Our proposal extends the c-variable method used in discrete-time decision process by introducing a new linear constraint for continuous time. The advantage of our approach is that it reduces the continuous-time Markov decision process to a discrete-time Markov decision process where the linear constraints make the problem computationally tractable. The usefulness of the method is illustrated in chemical reactions where the concentration dynamics is modelled as a continuous-time Markov chain. The first application is a single reversible reaction for the formation of the amidogen radical where we found the optimal temperature that minimizes the average expected rate of H formation at steady state. The second is a chemical reaction network for the proton transfer, hydration and tautomeric reaction of anthocyanin pigments, in this case we found an optimal strategy over a set of values of H ⁺ that minimizes the average expected total number of molecules at steady state.