Stochastic processes with expected stopping time

Chatterjee K, Doyen L. 2021. Stochastic processes with expected stopping time. Proceedings of the 36th Annual ACM/IEEE Symposium on Logic in Computer Science. LICS: Symposium on Logic in Computer Science, 1–13.


Conference Paper | Published | English

Scopus indexed
Author
Department
Abstract
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical optimization criterion is the maximal expected total reward where the MDP stops after T steps, which can be computed by a simple dynamic programming algorithm. We consider a natural generalization of the problem where the stopping times can be chosen according to a probability distribution, such that the expected stopping time is T, to optimize the expected total reward. Quite surprisingly we establish inter-reducibility of the expected stopping-time problem for Markov chains with the Positivity problem (which is related to the well-known Skolem problem), for which establishing either decidability or undecidability would be a major breakthrough. Given the hardness of the exact problem, we consider the approximate version of the problem: we show that it can be solved in exponential time for Markov chains and in exponential space for MDPs.
Publishing Year
Date Published
2021-07-07
Proceedings Title
Proceedings of the 36th Annual ACM/IEEE Symposium on Logic in Computer Science
Acknowledgement
We are grateful to the anonymous reviewers of LICS 2021 and of a previous version of this paper for insightful comments that helped improving the presentation. This research was partially supported by the grant ERC CoG 863818 (ForM-SMArt).
Page
1-13
Conference
LICS: Symposium on Logic in Computer Science
Conference Location
Rome, Italy
Conference Date
2021-06-29 – 2021-07-02
ISSN
IST-REx-ID

Cite this

Chatterjee K, Doyen L. Stochastic processes with expected stopping time. In: Proceedings of the 36th Annual ACM/IEEE Symposium on Logic in Computer Science. Institute of Electrical and Electronics Engineers; 2021:1-13. doi:10.1109/LICS52264.2021.9470595
Chatterjee, K., & Doyen, L. (2021). Stochastic processes with expected stopping time. In Proceedings of the 36th Annual ACM/IEEE Symposium on Logic in Computer Science (pp. 1–13). Rome, Italy: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/LICS52264.2021.9470595
Chatterjee, Krishnendu, and Laurent Doyen. “Stochastic Processes with Expected Stopping Time.” In Proceedings of the 36th Annual ACM/IEEE Symposium on Logic in Computer Science, 1–13. Institute of Electrical and Electronics Engineers, 2021. https://doi.org/10.1109/LICS52264.2021.9470595.
K. Chatterjee and L. Doyen, “Stochastic processes with expected stopping time,” in Proceedings of the 36th Annual ACM/IEEE Symposium on Logic in Computer Science, Rome, Italy, 2021, pp. 1–13.
Chatterjee K, Doyen L. 2021. Stochastic processes with expected stopping time. Proceedings of the 36th Annual ACM/IEEE Symposium on Logic in Computer Science. LICS: Symposium on Logic in Computer Science, 1–13.
Chatterjee, Krishnendu, and Laurent Doyen. “Stochastic Processes with Expected Stopping Time.” Proceedings of the 36th Annual ACM/IEEE Symposium on Logic in Computer Science, Institute of Electrical and Electronics Engineers, 2021, pp. 1–13, doi:10.1109/LICS52264.2021.9470595.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data IST Research Explorer

Sources

arXiv 2104.07278

Search this title in

Google Scholar
ISBN Search