---
_id: '1598'
abstract:
- lang: eng
text: 'We consider Markov decision processes (MDPs) with specifications given as
Büchi (liveness) objectives, and examine the problem of computing the set of almost-sure
winning vertices such that the objective can be ensured with probability 1 from
these vertices. We study for the first time the average-case complexity of the
classical algorithm for computing the set of almost-sure winning vertices for
MDPs with Büchi objectives. Our contributions are as follows: First, we show that
for MDPs with constant out-degree the expected number of iterations is at most
logarithmic and the average-case running time is linear (as compared to the worst-case
linear number of iterations and quadratic time complexity). Second, for the average-case
analysis over all MDPs we show that the expected number of iterations is constant
and the average-case running time is linear (again as compared to the worst-case
linear number of iterations and quadratic time complexity). Finally we also show
that when all MDPs are equally likely, the probability that the classical algorithm
requires more than a constant number of iterations is exponentially small.'
acknowledgement: "The research was supported by FWF Grant No. P 23499-N23, FWF NFN
Grant No. S11407-N23 (RiSE), ERC Start Grant (279307: Graph Games), and the Microsoft
Faculty Fellows Award. Nisarg Shah is also supported by NSF Grant CCF-1215883.\r\n"
article_processing_charge: No
author:
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
- first_name: Manas
full_name: Joglekar, Manas
last_name: Joglekar
- first_name: Nisarg
full_name: Shah, Nisarg
last_name: Shah
citation:
ama: Chatterjee K, Joglekar M, Shah N. Average case analysis of the classical algorithm
for Markov decision processes with Büchi objectives. *Theoretical Computer Science*.
2015;573(3):71-89. doi:10.1016/j.tcs.2015.01.050
apa: Chatterjee, K., Joglekar, M., & Shah, N. (2015). Average case analysis
of the classical algorithm for Markov decision processes with Büchi objectives.
*Theoretical Computer Science*, *573*(3), 71–89. https://doi.org/10.1016/j.tcs.2015.01.050
chicago: 'Chatterjee, Krishnendu, Manas Joglekar, and Nisarg Shah. “Average Case
Analysis of the Classical Algorithm for Markov Decision Processes with Büchi Objectives.”
*Theoretical Computer Science* 573, no. 3 (2015): 71–89. https://doi.org/10.1016/j.tcs.2015.01.050.'
ieee: K. Chatterjee, M. Joglekar, and N. Shah, “Average case analysis of the classical
algorithm for Markov decision processes with Büchi objectives,” *Theoretical
Computer Science*, vol. 573, no. 3, pp. 71–89, 2015.
ista: Chatterjee K, Joglekar M, Shah N. 2015. Average case analysis of the classical
algorithm for Markov decision processes with Büchi objectives. Theoretical Computer
Science. 573(3), 71–89.
mla: Chatterjee, Krishnendu, et al. “Average Case Analysis of the Classical Algorithm
for Markov Decision Processes with Büchi Objectives.” *Theoretical Computer
Science*, vol. 573, no. 3, Elsevier, 2015, pp. 71–89, doi:10.1016/j.tcs.2015.01.050.
short: K. Chatterjee, M. Joglekar, N. Shah, Theoretical Computer Science 573 (2015)
71–89.
date_created: 2018-12-11T11:52:56Z
date_published: 2015-03-30T00:00:00Z
date_updated: 2020-08-11T10:09:48Z
day: '30'
department:
- _id: KrCh
doi: 10.1016/j.tcs.2015.01.050
ec_funded: 1
external_id:
arxiv:
- '1202.4175'
intvolume: ' 573'
issue: '3'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: http://arxiv.org/abs/1202.4175
month: '03'
oa: 1
oa_version: Preprint
page: 71 - 89
project:
- _id: 2584A770-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: P 23499-N23
name: Modern Graph Algorithmic Techniques in Formal Verification
- _id: 25863FF4-B435-11E9-9278-68D0E5697425
call_identifier: FWF
grant_number: S11407
name: Game Theory
- _id: 2581B60A-B435-11E9-9278-68D0E5697425
call_identifier: FP7
grant_number: '279307'
name: 'Quantitative Graph Games: Theory and Applications'
- _id: 2587B514-B435-11E9-9278-68D0E5697425
name: Microsoft Research Faculty Fellowship
publication: Theoretical Computer Science
publication_status: published
publisher: Elsevier
publist_id: '5571'
quality_controlled: '1'
related_material:
record:
- id: '2715'
relation: earlier_version
status: public
scopus_import: 1
status: public
title: Average case analysis of the classical algorithm for Markov decision processes
with Büchi objectives
type: journal_article
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 573
year: '2015'
...