article
Average case analysis of the classical algorithm for Markov decision processes with Büchi objectives
published
yes
Krishnendu
Chatterjee
author 2E5DCA20-F248-11E8-B48F-1D18A9856A870000-0002-4561-241X
Manas
Joglekar
author
Nisarg
Shah
author
KrCh
department
Modern Graph Algorithmic Techniques in Formal Verification
project
Game Theory
project
Quantitative Graph Games: Theory and Applications
project
Microsoft Research Faculty Fellowship
project
We consider Markov decision processes (MDPs) with specifications given as Büchi (liveness) objectives, and examine the problem of computing the set of almost-sure winning vertices such that the objective can be ensured with probability 1 from these vertices. We study for the first time the average-case complexity of the classical algorithm for computing the set of almost-sure winning vertices for MDPs with Büchi objectives. Our contributions are as follows: First, we show that for MDPs with constant out-degree the expected number of iterations is at most logarithmic and the average-case running time is linear (as compared to the worst-case linear number of iterations and quadratic time complexity). Second, for the average-case analysis over all MDPs we show that the expected number of iterations is constant and the average-case running time is linear (again as compared to the worst-case linear number of iterations and quadratic time complexity). Finally we also show that when all MDPs are equally likely, the probability that the classical algorithm requires more than a constant number of iterations is exponentially small.
Elsevier2015
eng
Theoretical Computer Science
1202.417510.1016/j.tcs.2015.01.050
573371 - 89
https://research-explorer.app.ist.ac.at/record/2715
Chatterjee, Krishnendu, Manas Joglekar, and Nisarg Shah. “Average Case Analysis of the Classical Algorithm for Markov Decision Processes with Büchi Objectives.” <i>Theoretical Computer Science</i> 573, no. 3 (2015): 71–89. <a href="https://doi.org/10.1016/j.tcs.2015.01.050">https://doi.org/10.1016/j.tcs.2015.01.050</a>.
Chatterjee, K., Joglekar, M., & Shah, N. (2015). Average case analysis of the classical algorithm for Markov decision processes with Büchi objectives. <i>Theoretical Computer Science</i>, <i>573</i>(3), 71–89. <a href="https://doi.org/10.1016/j.tcs.2015.01.050">https://doi.org/10.1016/j.tcs.2015.01.050</a>
Chatterjee K, Joglekar M, Shah N. Average case analysis of the classical algorithm for Markov decision processes with Büchi objectives. <i>Theoretical Computer Science</i>. 2015;573(3):71-89. doi:<a href="https://doi.org/10.1016/j.tcs.2015.01.050">10.1016/j.tcs.2015.01.050</a>
K. Chatterjee, M. Joglekar, N. Shah, Theoretical Computer Science 573 (2015) 71–89.
K. Chatterjee, M. Joglekar, and N. Shah, “Average case analysis of the classical algorithm for Markov decision processes with Büchi objectives,” <i>Theoretical Computer Science</i>, vol. 573, no. 3, pp. 71–89, 2015.
Chatterjee K, Joglekar M, Shah N. 2015. Average case analysis of the classical algorithm for Markov decision processes with Büchi objectives. Theoretical Computer Science. 573(3), 71–89.
Chatterjee, Krishnendu, et al. “Average Case Analysis of the Classical Algorithm for Markov Decision Processes with Büchi Objectives.” <i>Theoretical Computer Science</i>, vol. 573, no. 3, Elsevier, 2015, pp. 71–89, doi:<a href="https://doi.org/10.1016/j.tcs.2015.01.050">10.1016/j.tcs.2015.01.050</a>.
15982018-12-11T11:52:56Z2020-08-11T10:09:48Z